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Foreword 


Maurizio Rossi, Davide Spallazzo 
Department of Design, Politecnico di Milano 


The relationship between design and the digital world has pro- 
gressed in two directions since the dawn of developing the first com- 
puter systems. The first phase of this relation saw the digital world 
become the object of design, in terms of forms, algorithms, and inter- 
action methods. With the advent of CAD systems, the digital world 
has proposed itself as a design support tool at various levels of scale, 
from the design of industrial products to interiors up to the scale of 
large infrastructures. Today we are witnessing the full development of 
the biunivocity of the relationship between design and the digital 
world. The latter simultaneously supports design and is the subject of 
designing the behavior for morphological and systemic elements of 
modern products and buildings. 

In the context of the processes concerning digitally enhanced de- 
sign, we should mention the new frontier of computational design, 
which consists of applying computational strategies to the design pro- 
cess. These issues have also been brought into building automation 
and smart home in architecture, where interaction design has devel- 
oped based on the IoT. The behavior of the new building’s complex 
systems, and even its morphology, may depend on the collection and 
management of data deriving from the behavior of people and the 
changing environmental conditions inside and outside the building. In 
the context of architecture, one of the classic examples is the control 
of natural light to enhance it but keep it within acceptable parameters 
for the needs and lifestyles of the interiors. Thanks to the IoT, artificial 
lighting, HVAC, and audiovisuals are now connected to new building 
automation. The new frontier is no longer just automation but a new 
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level of intelligent interaction, AI-infused systems able to learn over 
time and modify their behavior accordingly. 

Today sensors, increasingly pervasive and connected, can provide 
different information. This info regarding the occupants and the indoor 
and outdoor are sent to robotic management systems capable of learn- 
ing from human behavior and natural environment variations. The pur- 
pose is to set up integrated systems of the building for people’s well- 
being and energy saving. Sensors have come to the point of using com- 
puter vision for the analysis of environments and user behaviors. How- 
ever, in democratic countries, these systems find little acceptance by 
users, especially in the workplace, when they allow workers to be 
identified and monitored. Next to the video sensors, thermopile array 
systems are being developed that would enable the analysis of the en- 
vironment and the activities that take place there in the infrared field, 
thus ensuring privacy. If solutions to protect personal data while en- 
suring full operational capability are emerging, scenarios of domestic 
environment integrating AI solutions may emerge and be discussed on 
the one side as utopia/dystopia for end-users and as opportunities for 
designers on the other. 

This kind of innovation leads the design and management of spaces 
to become a time-based process. This raises important questions re- 
garding the representation and formalization of the project for its real- 
ization, management, and future maintenance of complex, integrated, 
interactive systems that should be able to learn. The formalization of 
projects with this degree of descriptive complexity, no longer only for- 
mal but also managerial, can be tackled today only thanks to Building 
Information Modeling methods. 

While BIM represents the future in the design methodology of 
buildings, the digital world tools available today cannot provide all the 
solutions necessary to manage interior design. This area lacks funda- 
mental tools to manage color and lighting design in a reliable way be- 
tween the digital and real worlds. All the digital world tools should be 
integrated, reliable, and supportive in the design process. Through cal- 
culations and photorealistic representation, they should allow virtual 
prototyping of spaces. Only in the final phase can provide helpful tools 
for the presentation and multimedia communication of the project that 
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must not be confused with those of the beautiful images typical of 
communication design. 

There is an open problem concerning the importation into the digi- 
tal world of the colors and lighting of reality. On the other hand, there 
is also a problem concerning the correctness of colors and lighting de- 
sign in the digital world when forecasting the real world. These prob- 
lems arise from the fact that in the physical world, as in the represented 
digital world, there is the human being with visual perception, which 
is a cognitive process that has not yet been described in all its aspects. 
Do not deceive the presence of sciences such as photometry and col- 
orimetry because these historically underestimate the importance of 
real visual perception, well known in the design practice since the 
Bauhaus times. Our optical system does not work as a precise meas- 
urement tool or a camera with the image that reaches our eyes; instead, 
it is based on a process of spatial perception not yet fully described. 
Each visual element depends on the context. This is even more evident 
in the digitization of cultural assets, for their cataloging, safeguarding, 
and enhancement, which are central to design for cultural assets. For 
historiographic reconstruction and enhancement of the work of art, the 
level of acquisition in the digital world has arrived at the analysis of 
the spectral and non-visible radiation, which has allowed the reading 
of the understated drawings in paintings and the studying of unreada- 
ble scripts. 

On a lower scale, in the modern design process, in the manufactur- 
ing sector through CAM and digital numerical control robots, new 
smart materials are now available, but an environmental sustainability 
problem remains open. The integration achieved between conven- 
tional design materials and new technologies of the digital world has 
led to the definition of a new class of materials, which we can define 
as phygital, with dynamic behaviors thanks to their ability to straddle 
the physical and the digital world. It is also thanks to a recovery of the 
aesthetic value of the project, accompanied by an experimental rein- 
terpretation of basic design, that these materials, changing in shape 
and appearance, now give new degrees of freedom to the design of 
industrial products. 

Alongside aesthetics, however, there is an important open issue of 
design ethics in the furniture sector. We know the values of the circular 
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economy cycle, as reinforced by the European Green Deal launched 
in 2019. An enormous accumulation of non-recyclable waste charac- 
terizes the unbridled consumerism of today’s capitalist system. Unfor- 
tunately, European data show that only 10% of furniture products are 
recycled in the supply sector while 90% ends up in landfills. In this 
context, the digital tools of Industry 4.0, such as the management of 
data flow from the 3D project to the product, smart robots, 3D printing, 
autonomous handling systems, and intelligent energy management, 
can facilitate the transition towards a factory based on principles of 
renewable and clean energy to ensure eco-friendly production. 

This book recalls theories and experiments in the relationship be- 
tween the physical and digital world with a multidisciplinary approach 
and different methodologies typical of design research and practice, 
placing itself in an open research path with further developments in 
the coming years. 


10 
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1. New scenarios for lighting design tools 


Andrea Siniscalco, Gianluca Guarini 
Department of Design, Politecnico di Milano 


Abstract 


The lighting project comprises several steps, each of which in- 
volves using tools (and skills) that can vary in the process phases. A 
designer should know the fundamentals of lighting to design in full 
awareness, but numerous software tools can provide valuable help be- 
yond this. The negative aspect of this variety is that heterogeneous 
outputs can be obtained due to individual designers’ skills and prefer- 
ences. Although diversity is a value to be pursued, it is also true that 
the results must still be correct. For example, the most common three- 
dimensional modeling software are not explicitly designed to take care 
of the lighting project. By contrast, software specializing in lighting 
design often provides results that, although correct from the point of 
view of the calculations, are inadequate in reproducing reality. An al- 
ternative is to adopt a Building Information Modeling (BIM) type of 
software, manage complexity, and integrate them with other programs 
that can somehow interface. This solution allows managing the project 
to comply adequately with the directives that promote BIM as a man- 
datory methodology for future projects. This hybrid approach, albeit 
imperfect, is functional, hoping that specialist software developers and 
product companies (for their catalogues) will quickly adapt to the BIM 
system. While waiting for a perfectly integrated methodology, some 
fundamental problems for lighting design must be addressed. 

The reproduction of the products and their associated characteris- 
tics (outside the context of specialized software) is not easy to achieve. 
The correct simulation of the finishes, which involves implementing 
standard procedures to acquire the project’s colors digitally, is another 
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topic that needs further discussion. This chapter describes original so- 
lutions at the Light Lab of Politecnico di Milano to search for the best 
method to achieve photorealistic and photometrically correct results. 


1.1 Introduction 


Lighting design is a professional field that requires a well-defined 
set of expertise. Most lighting designers are trained as architects, but 
it is not uncommon to find professionals from electrical or electronic 
engineering studies or design-oriented faculties. However, the absence 
of a chamber of lighting designers (mostly belonging to various asso- 
ciations) results in a significant heterogeneity in the approach to the 
project. Numerous differences can be found in lighting projects. First, 
the application field already determines distinctions in the deliverables 
that are produced. For example, a project for works in the public ad- 
ministration area requires more detailed documentation than the pri- 
vate one. Other differences may arise from the design context; the 
lighting of an interstate road requires calculations, simulations, and 
documents that differ from those for cultural heritage, retail, hospitals. 

In addition to these differences, the competencies of the lighting 
designer also affect the approach to the project; some designers are 
more interested in the expression of the concept underlying the project, 
while others focus on energy efficiency or in solving specific technical 
problems. Finally, and this happens above all in the private sector, the 
most disparate requests of the client translate into the variability of 
approaches, which are potentially different for each project. 

Although diversity and novelty in the solution of design challenges 
are considered values, it should also be assumed that it is necessary to 
realize a project at the end of this flow of ideas. Projects must be fea- 
sible; they must define a series of lighting values, power needs, elec- 
trical consumption, number of specific lighting fixtures, positioning, 
aiming, etc. To facilitate designers to follow a single direction in the 
project, in 2016, the Italian Standardization Body produced a standard 
(UNI, 2016) whose purpose is to unify the lighting design process and 
the related documentation. This standard, which has no equal in Eu- 
rope, provides definitions, procedures and the material that must be 
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delivered to the client. However, the norm does not go into the details 
of specific tools to be used. As regards the graphical drawings, it re- 
mains very open to the type of data representation. Terms such as 
“photo simulations” or “graphic rendering in an appropriate scale” 
are used, but without going into details on the method of production 
of these elements. Consequently, each designer is free to adopt the 
techniques he deems appropriate to simulate the project’s appearance, 
from the production of photo simulations with raster imaging software 
to the use of 3D applications for the production of photorealistic ren- 
derings. In the standard, emphasis is instead placed on the possibility 
of using lighting calculation software as long as they are validated. 
There are numerous software (or, more precisely, calculation algo- 
rithms) that allow one to simulate the interaction with matter to pro- 
duce photorealistic images. Still, the type of programs mentioned in 
UNI 11630 does not refer to the ability to create such images, as much 
as instead to the possibility of serving as a support for the lighting cal- 
culation. 


1.2 The calculation of lighting 


Since the appearance of three-dimensional modeling software, re- 
search has been concerned with simulating the lighting on objects cor- 
rectly. At first, virtual sources were used, known as OpenGL (Point 
light, Spot light, Ambient light, Directional light). These virtual lights 
were designed best to mimic the types of sources present in nature, so 
the spotlight simulated the cones of light, directional light served as 
sunlight, ambient light emulated indirect light, etc. 


» 


Point light Spot light Ambient light Directional light 


\\/ L 
‘i Me RY 
a n me 


Fig. 1.1 - From left to right, different generic light sources commonly used in 3D 
software: Point, Spot, Ambient and Directional. 
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The render engines worked with local computation models (flat 
shading, Gouraud shading, etc.) and ray-tracing (Goldstein and Nagel, 
1971). With these methods, it was possible to represent glossy and re- 
flective surfaces, transparencies and sharp shadows. Still, it was im- 
possible to calculate the diffused light, which was mimicked by the 
ambient light (assigning a specific value to a “fill light”). Neverthe- 
less, it was possible to obtain digital images that were quite plausible 
with this method even though they were far from what reality would 
have been by reconstructing the space with actual lighting products. 

As time passed, the quality of the simulations improved. Software 
packages began to introduce the radiosity algorithm (Goral et al., 
1984), which allowed the simulation of the interaction of light with 
diffusive surfaces. By combining radiosity and ray-tracing, the first 
software oriented to lighting analysis began to appear (such as 
Lightscape, in 1994). These applications were capable of providing 
not only increasingly photorealistic renderings but also improving 
photometric accuracy. Further improvements have come with more 
recent engines (Ng et al., 2001), such as photon mapping (Jensen, 
2001) or the physically-based approach (Pharr, Jakob and Humphreys, 
2017; Chen, Cui and Hao, 2019). The incredible complexity of the 
light-matter interaction makes it necessary to apply simplifications in 
the calculation, which otherwise would result in timing not compatible 
with the reality of professional practice. Despite this, however, the cal- 
culation engines available today make it possible to obtain a satisfac- 
tory lighting simulation to support the lighting project. This result is 
also possible because instead of OpenGL lights, photometric files ob- 
tained from the measurement of existing luminaires are used. 


1.3 Representation software vs Lighting design software 


Even if the render engines used are the same, it is possible to dis- 
tinguish between the software used only to produce photorealistic im- 
ages and those used to support the lighting design. This distinction 
depends on various factors, but the main reason remains the intended 
use of the software. The production of a “photorealistic” image of an 
environment (for example, the simulation of an architectural project) 
aims to promote the solutions designed to the client. A beautiful image 
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helps the designer to sell better his project, and in this context, the 
lighting analysis tools are not essential. Sometimes it is also possible 
to “cheat” by tweaking some parameters to make the image even more 
attractive (more saturated colors, lens flares, halos, materials more re- 
flective than they should). The use of lighting design software sees its 
strength in the presence of lighting analysis tools. For example, these 
software packages can give feedback on the lighting levels on specific 
surfaces or evaluate the uniformity of illuminance on a visual task, 
rather than the modeling factor on the occupants’ faces, etc. Rendering 
is a secondary feature, and in any case, it is always subordinate to the 
photometric values obtained. There are no “embellishments”, as they 
would then be refuted after constructing the actual space. 


et 


Fig. 1.2 - A render generated with 3D modeling software. The purpose of the image 
is to promote the solution designed by the architect. Therefore, the image is realistic, 
but some errors are visible; the interior/exterior balance of luminances and the 
sources reflections on the windows (too visible for a daylight scene). 
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Fig. 1.3 - Render of a lighting software. The image is photometrically accurate, but 
the poor management of the 3D geometry causes a loss in photorealism 


Fig. 1.4 - Non-photorealistic render generated by lighting software. This type of 
image (coupled with a proper color scale) acts as a tool to check the correct illumi- 
nance values in the space. This allows an accurate idea of the amount of light on the 
surfaces regardless of image exposure levels. 
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Fig. 1.5 - Rendering obtained using a photo-editing program. Starting from daytime 
images, one can recreate a night scene by adding layers of black. The images thus 
obtained, however, usually lack photometric accuracy. 


Calculation grids, verification points, non-photorealistic rendering, 
automatic production of verification reports are the primary tools of 
this kind of software. 

In order to be considered reliable, a lighting calculation software 
must be “validated”, that is, it must comply with the criteria set out in 
the technical document 171:2006, produced by the technical commit- 
tee 3.33 of the Commission Internationale de l’Eclairage (CIE, 2006). 
However, this step is not required for 3D representation software. 

The most popular lighting design software are Agi32 (Lighting An- 
alysts, 2021), Dialux (DIAL GmbH, 2021), Relux (Relux Informatik 
AG, 2021), Litestar (Oxytech, 2021). 

Generally, lighting calculation software is not particularly complex 
in terms of modeling, texturing and material management tools. This 
makes the time required to master them considerably less than that of 
representation software but causes apparent limitations in the 
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generation of photorealistic images. Moreover, even if the calculation 
algorithms are comparable (though often simplified), the vertical soft- 
ware (lighting design) are not efficient in the management of complex 
geometries. 

The direct consequence of the lack of photorealism in verification 
software is that when designers want to obtain high-quality images 
(for large commissions or big tenders, for example), they find them- 
selves having to resort to other tools. These might be the 3D modeling 
programs themselves or other solutions, such as the photo editing of 
pictures with raster editing programs. 

The problem of using multiple tools (other than the increase in the 
time and resources required) is the difficulty in obtaining uniform re- 
sults between different instruments. By using various programs, the 
consistency of the outputs is often determined by the skills and expe- 
rience of the designer. 


1.4 Building Information Modeling 


The lack of uniformity in the production of the documents, espe- 
cially in the construction sector, can lead to significant difficulties, es- 
pecially in larger projects. These differences can lead to misinterpre- 
tation of the data by the various work teams, which often causes de- 
lays, if not errors. 

To stem this problem, since 2014, the European Parliament (Di- 
rective 2014/24/EU of the European Parliament and of the Council of 
26 February 2014 on public procurement and repealing Directive 
2004/18/EC Text with EEA relevance, 2014) has proposed (art. 22, 
sub. 4) that works of public interest be managed through the use of 
Building Information Modeling systems. 

To limit the lack of consistency in the data of the lighting project, 
this too should be integrated within the scope of the BIM methodology 
(Guarini, 2020). 

At the moment, this choice appears to be ideal for establishing 
guidelines for two reasons, the first relating to the specifically collab- 
orative nature of this procedure and the second to the regulatory im- 
position on its use, which is increasingly growing in recent years, and 
it will be even more so in those to come. 
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Concerning the first reason, Information Modeling has as its fun- 
damental principle the collection and sharing of information relating 
to the functions of the elements of the project. This information is 
available to all professionals in the different sectors involved, archi- 
tects, structural engineers, plant engineers. Among these, lighting de- 
signers can also benefit from this sharing of information and, at the 
same time, contribute to the work of other studios. However, there is 
a problem related to the current state of maturity of the BIM method- 
ology, not yet at its maximum level. A diagram developed by Mark 
Bew and Mervyn Richards (Succar, 2015) shows four levels of BIM 
Maturity (3+1). The chart shows the ability/possibility of the de- 
sign/production chain to exchange digital information relating to the 
building is measured under construction. 


e Level 0 is characterized by the total absence of collaboration. The 
project is made only through 2D images, disclosed and consulted 
only through printed tables or pdf (or both versions simultane- 
ously). It represents the first step from analogue drawing (drafting 
table) to digital drawing (CAD). 

* Level I adds the 3D dimension to the project drawings, but the 
sharing of information is minimal. These are disclosed only as pa- 
per or vector output, but the 3D model that generated them remains 
with the designer, who does not share it in any way with other op- 
erators. The documentation produced is mainly used for approval 
by the administrations. 

e Level 2 is the actual passage to the BIM procedure. All parties in- 
volved use their specific 3D model, but this time the information 
can be shared via export with a standard (ISO, 2018) file format 
* IFC (Industry Foundation Classes) (BuildingSMART, 2021). 
Other professionals can import this file type into their software in 
a consultative manner, that is, without modifying it but using it to 
check for any geometry conflicts and extrapolate the information 
of the individual construction elements. A limit for this BIM level 
is the need for a new export/import if there’s a change to the orig- 
inal design. Nevertheless, this is the most common level of opera- 
tion today, and lighting designers can benefit from this approach, 
albeit with the inherent limitations of this degree of maturity. 
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e Level 3 is BIM in its full maturity as a collaborative methodology. 
Each designer works on a single shared model, which can also be 
consulted and modified by the other actors involved. This level has 
the undoubted advantage of constantly updating and validating 
data without the need for continuous exports and imports. All this 
presupposes the use of a single standard format. Unfortunately, 
due to the different software houses and the different disciplines 
involved, it is difficult to determine what this format may be, es- 
pecially for commercial and copyright reasons. Due to these is- 
sues, this level has currently only been hypothesized and has not 
yet been reached. 


The choice of opting for a BIM solution is, secondarily, dictated by 
the current regulatory framework (standard ISO/CD 19650:1-4), 
which will come into force after an initial legislative gap. Numerous 
states of the European Union have activated a roadmap that will lead 
to the mandatory nature of BIM practices in the context of tenders (EU 
BIM Task Group, 2018). 

The Italian government, for example, has dictated through the so- 
called “BIM Decree” D.M.560 of 1.12.2017 (Ministero delle Infra- 
strutture e dei Trasporti, 2017) the time frame for the introduction of 
the mandatory use of this methodology in public works contracts that 
present an amount higher than a certain amount (standard UNI 
11337:1-10). The application of the law was valid for projects over 
100 million euros on 01 January 2019. The threshold is expected to 
decrease to | million euros in 2023, while from 2025, the obligation 
will be valid for all new public works, regardless of the amount. 


1.5 Lighting in BIM software 


Most software that implement BIM logics can read photometric 
files. As already mentioned, to use the software for project support, it 
is necessary to use virtual sources that can reproduce the correct dis- 
tribution of light intensities (and not just represent an ideal source, as 
for OpenGL lights). These files report angular coordinates and the cor- 
responding luminous intensity value. In doing so, it is possible to de- 
scribe how light behaves in space. 
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In order to obtain these files, it is necessary to measure the lumi- 
nous intensities of the luminaire (generally on an instrument called a 
goniophotometer). The measurement implies the use of a coordinate 
system (CEN, 2012). It is a set of planes with a single axis of intersec- 
tion (polar axis) on which the photometric center of the luminaire lies. 
Two angles are required to determine a direction in space: the one be- 
tween the reference plane and the following half-planes and the one 
between the polar axis and the direction considered. Generally, the 
most used reference system (especially in interiors) is called C-y. 


Fig. 1.6 - The C-y reference system as described by the standard EN 13032-1: 2012. 
1) Primary axis (polar). 2) Secondary axis. 3) Tertiary axis. 4) C-plane. The yellow 
dot represents the photometric center of the luminaire. 


There are numerous formats for photometric files. The most uni- 
versally accepted by many software packages is the IES file format, 
developed by the Illuminating Engineering Society (IES, 2021) and 
regulated by a specific standard, the ANSI/IES LM-63-19. 

This type of file is generally read by BIM software, to name the 
most common: Autodesk Revit (Autodesk, 2021b), Graphisoft Ar- 
chicad (Graphisoft, 2021) and Allplan (Nemetschek, 2021). However, 
there remains a problem related to the representation of the lighting 
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fixture. Photometric files such as the IES do not contain detailed in- 
formation regarding the geometry of the luminaire, representing it as 
a generic cube or cylinder capable of emitting light. In order to recreate 
a scene where lighting products are represented, this can be an issue. 
One can associate the photometric file with a 3D geometry that repli- 
cates the luminaire features. In that case, it is not uncommon to place 
the photometric center within the 3D geometry itself. The problem 
arises from the fact that, in this case, the distribution of light intensity 
could be hindered by the geometry, significantly interfering with the 
correct photometric simulation to be obtained (Siniscalco and Guarini, 
2018). For example, some CAD software - like 3D Studio Max (Au- 
todesk, 202 1a) - contemplate this problem and offer a solution to avoid 
this interaction. There is the possibility of preventing the geometry 
from casting shadows when it is illuminated. In the case of BIM soft- 
ware, this solution is not available. However, it is possible to use plug- 
ins that solve the problem (in a different way) at the level of the ren- 
dering engine. Instead of excluding geometry from the projection of 
shadows, these plug-ins exclude the materials assigned to the lumi- 
naire’s geometries. In order to operate this solution, the luminaires ma- 
terial must not be native to the BIM software but specific to the plug- 
in. An example of this approach is that of VRay for Revit (Chaos 
group, 2021). 


1.6 The color of materials 


A further problem to be faced in an attempt to reproduce the inter- 
action between light and matter correctly is the Colour, Material and 
Finish (CMF) design simulation. Faithful reproduction can be 
achieved by using multiple devices, both input and output. This plu- 
rality of tools raises the problem of Color Consistency, which can be 
solved using Color Management (Rodney, 2005). 

Acquiring the color of the same material with a scanner or a camera 
could lead to different digitization. Similarly, the display of the results 
of this acquisition can be perceived differently by the observer if this 
is done through a monitor or printer. It could be perceived differently 
even between two monitors of the same brand and model but with dif- 
ferent settings. It can therefore be said that each hardware device has 
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its own color profile. Furthermore, an acquisition device should be 
periodically calibrated to obtain a valid color profile. This because of 
many factors, like the non-immutability of the acquisition perfor- 
mance of the device. 

The methods for performing this calibration can change between 
different devices. For example, for a camera, the procedure consists of 
framing a standard color palette within a shot relating to the finish. An 
example of these is given by X-Rite (X-Rite, 2021a). Through propri- 
etary software or plug-ins for the most used photo-editing programs, 
it is possible to code the colors present in these palettes positioned 
within the shots and obtain the device’s color profile associated with 
a specific lighting condition. 

If the acquisition has to be done with a scanner, the procedure 
would consist of using another type of color table, like the ANSI IT8 
(Coloraid.de, 2021) or the ISO 12461 (ISO, 2016). This calibration 
implies the color chart’s acquisition without the actual sample that 
must be acquired. This is because, in the case of the scanner, the light- 
ing conditions of the surrounding environment do not influence the 
result. To achieve that, the sample to be scanned must be kept in a 
standardized situation, like as inside the cover of the scanner which 
will be placed under a black cloth. Subsequently, a specific software 
generally supplied with the IT8 sample - like VueScan (Hamrick, 
2021) - can generate the scanner’s color profile. 

The monitor on which the acquisition results are observed should 
also be periodically calibrated. In order to do this, there are various 
hardware solutions - like the SpyderX (Datacolor, 2021) or the Col- 
orMunki Smile (X-Rite, 2021b) - with related software that guides the 
user in the calibration process. 

Once the color profiles of the various devices have been obtained 
(called relative because they refer to specific hardware), the color con- 
sistency is obtained by making transformations between the various 
profiles and an absolute color space, independent of the device used. 

The universal (and often unique) absolute color space managed by 
CAD software (and also the BIM ones) is the sRGB color space (Niel- 
sen and Stokes, 1998). Therefore, to use colors as faithful as possible 
to those of the finishes, it is advisable to perform these transformation 
operations between the relative profiles and the absolute sRGB profile 
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in photo editing software with an efficient and complete color man- 
agement module. 


1.7 Conclusions 


Having found the best procedures to faithfully reproduce lumi- 
naires and the distribution of their luminous intensities and the color 
of the materials, it should be possible to produce images that are pho- 
tometrically correct and also photorealistic. Furthermore, this goal 
should be achievable without worrying about embellishing the pic- 
tures with tweaks that would be denied during the realization of the 
project, causing a possible dispute with the end customer. 

Lighting design software, as already mentioned, while providing 
photometrically correct results, do not generally reach adequate pho- 
torealism levels. The software commonly used in BIM procedures 
have the potential to satisfy both photometric and photorealism re- 
quirements (Murdoch, Stokkermans and Lambooij, 2015). However, 
working in a BIM level 2 context and considering the collaborative 
nature (between different disciplines) of the methodology, a hybrid 
solution is usable. It is possible to leave the photorealistic rendering to 
the BIM modeling software, using specific plug-ins, and have the pho- 
tometric verification and compliance with standards to the lighting de- 
sign software. 

The dialogue between the two software is achieved by exporting 
the BIM work file in the * FC interchange format, subsequently im- 
ported into the lighting CAD. 
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Fig. 1.7 - Photorealistic image obtained directly in the BIM program with the help 
of a rendering engine plug-in. 


A limitation to this approach is that it is impossible to make the 
most of the tools of the two types of software. Thus, for example, light- 
ing design does not benefit from the interactive phase of the BIM 
methodology. At each modification of the architectural project, one is 
forced to modify the *.IFC file and redo the positioning of the lighting 
fixtures and other tools in the lighting software. Vice versa, once the 
lighting verification is done, to integrate the result into the architec- 
tural project, it is necessary to edit the BIM model by hand, reproduc- 
ing the solution verified in the lighting CAD. In consideration of this, 
the design and verification of lighting would risk being relegated to 
the final stage of the process rather than evolving with it. 

The continuous development of BIM software has led to the ap- 
pearance on the market of numerous plug-ins capable of providing so- 
lutions. Concerning the aspects related to lighting, we can mention the 
cloud-based solution LightStanza (2021). 

These plug-ins allow you to export the BIM model/database to the 
cloud by accessing it from a web browser. Thanks to this approach, 
the lighting design can be based on the luminaires included in the BIM 
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database or possibly introduce new ones. In addition to the calcula- 
tions (always in the cloud, decreasing the load on the computer), an- 
other novelty is the possibility of re-exporting the lighting solutions 
(calculations and documentation) from the lighting software to BIM 
in a simplified way. 

In any case, given the high complexity of the problem and the tools 
involved, it is easy to understand that currently, a single and complete 
solution for the integration of the lighting project in the BIM method- 
ology does not yet exist. But we expect, in the coming years and with 
the evolution of technologies, convergence towards a smaller and 
more efficient set of tools. 
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Abstract 


Color plays a fundamental role in cultural heritage applications. 
From frescoes to statues, from paintings to architectures, from photog- 
raphy to films, color is used to engage the public, communicate a mes- 
sage, and as a mean of expression of the artists. Although, the color 
sensation and perception derived from an object of cultural importance 
depend on the properties of the materials composing it, together with 
the spatial arrangement in which the colors are inserted and observed. 
When thinking about real objects, color is not just an esthetical ele- 
ment or only a mean of expression, but it is the consequence of the 
radiation-matter interaction, which is strictly related to the physical 
and chemical properties of the material. In this context, modern color- 
imetry became a fundamental science since the beginning of the last 
century, when it was first applied to industrial aims. 

Moreover, since Human-Computer Interaction and computational 
power are becoming more and more essential in our daily lives, greater 
importance is given to the digital color representation and to the digital 
color imaging. In this way the development of instruments, measure- 
ment strategies, and tools that can correctly reproduce and represent 
materials, contrast, and colors as faithful as possible to the original 
objects is fundamental. The correct reproduction of the color of an ob- 
ject of cultural importance is necessary for many applications (e.g., the 
analysis of physical color, the light design study, etc.) besides the res- 
toration, preservation, and valorization of the artwork itself. 
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In this essay, we will analyze the main limits of colorimetry as well 
as the current analytical techniques employed to digitally acquire the 
color information of physical objects for cultural heritage applications. 
We will focus on digital color reproduction issues, describing the im- 
portance and the difficulty of a correct illumination, and showing some 
practical examples of problems encountered in the color digitization 
workflow of physical color. 


2.1 Introduction 


The color of objects exists because of the interaction of three fac- 
tors: the light source illuminating the object, the physical and chemical 
properties of the material composing the object itself, and the response 
of the Human Visual System (HVS) (Oleari, 2015). In this triangle, 
the objects are visible because of the wavelengths they absorb, reflect, 
or transmit depending on their chemical and optical properties. Then, 
color is created by our vision system through two mechanisms: the 
photons capture by cones (also known as quanta catch) and the spatial 
computation of the receptors’ responses. Finally, the light source not 
only interacts with the object but also with the HVS, as it plays a vital 
role in the color appearance of objects through adaptation. 

The science attempting to physically describe the human color per- 
ception with an objective numerical system is colorimetry. It was first 
introduced by Maxwell in 1862 and from the beginning of the current 
century it has played an important role in all areas that involve color 
generation, perception, and rendition. This is even true in the field of 
cultural heritage, where the introduction of colorimetry has allowed a 
multiplicity of applications. 

Over time, if on one hand the spread of interest in the study of cul- 
tural heritage has led to the development of new non-invasive and non- 
destructive technologies for the analysis of materials, on the other 
hand this was not the case for color analysis. The colorimetric ap- 
proach is still stuck to the traditional analysis since the beginning of 
the twenty-first century: conservation scientists exploit it only to as- 
sess the color of the materials studying their optical properties and 
without considering the psycho-physical aspects of color vision. In 
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this way, color science still presents numerous open issues that have 
not been resolved yet. 

The spread of the wide variety of digital technologies deeply con- 
tributes to increasing the application of colorimetry in the field of cul- 
tural heritage as well. The digital color reproduction of the objects of 
cultural interest to enable their preservation and valorization is the 
most common purpose in this context. The digitization of the archives 
is also becoming more and more prevalent and is highlighted by vari- 
ous national and international digitization campaigns. In this field, the 
complexities of a correct color acquisition and reproduction are largely 
well-known, but this becomes particularly dangerous and challenging 
in the case of cultural heritage. 

Starting from the above considerations and aiming to grow aware- 
ness on the readers of this book, we organized the chapter into three 
sections: the first one presents a concise critical overview of some 
open problems in color characterization and measurements, the second 
part briefly presents the variety of color acquisition technologies and 
recording system; the third part introduces the most common problems 
encountered during digitizing cultural objects, describing the im- 
portance of instrument calibration, the difficulty of a correct acquisi- 
tion due to several open issues (such as glare phenomenon, the impos- 
sibility to obtain a uniform illumination, the complexity of a real scene 
and its bidimensional representation), and the difficulty of proper 
color management. 


2.2 Color formation and colorimetry 


The concept of color is often entangled with colorimetry. To better 
understand how a correct description, measurement, and evaluation of 
color is achieved, it is necessary to briefly introduce the principles of 
colorimetry and color formation. 

Two clear and complete critical overviews about these topics are 
(Rizzi, 2021a) and (Rizzi, 2021b). 

In his works, Rizzi underlines that the HVS generates color sensa- 
tion with two mechanisms: the photons capture by cones (i.e., the 
spectral radiance of a stimulus) and the spatial computation of the 
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receptors’ stimulation (i.e., the spatial arrangement of all the spectral 
radiances in our field of view). This means that color is not the unique 
product of retinal cone responses, since the local structures, such as 
edges and gradients, in the content of a scene change the final color 
sensation. 

Since the high complexity of the HVS, color science has simplified 
the color sensation characterization limiting the effect of spatial mech- 
anism. As a result, colorimetry works under the strong basic constraint 
of aperture mode, where a small color stimulus is isolated, and it is 
observed in a darkroom with no light. Based on this assumption, CIE 
XYZ is a pointwise colorimetric model in which the light responses of 
several stimuli do not interact with each other. This is deeply far from 
real situations and the contribution of the illuminant source and the 
objects are not distinguishable (see the scheme of Fig. 2.1). 


kg 


@ 
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Fig. 2.1 - Aperture mode constraint: a white source illuminating a yellow paper (left) 
and a yellow source illuminating a white paper (right) generate the same color sig- 
nal (Rizzi, 2021b). 


This ambiguity is solved with the so-called “perceptual uniform 
spaces”: CIE L*a*b* and CIE L*u*v*, which work in object mode (or 
surface mode), with the introduction of a reference illuminant. In this 
way it is possible to distinguish the exact color of a surface from the 
lighting source. Nevertheless, the color is still evaluated pointwise 
since spatial interaction is still not considered. 
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Fig. 2.2 — Example of visual illusions determined by the spatial interaction mecha- 
nism: simultaneous contrast (top), assimilation (center) and Kofka ring (bottom). 
(Rizzi, 2021a; Rizzi, 2021b; Koffka, 1935). 
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Fig. 2.2 shows some visual illusions that Rizzi presents in (Rizzi, 
2021a) and (Rizzi, 2021b) as examples of the effects of spatial ar- 
rangement on color perception. On the top, the simultaneous contrast 
phenomenon, that changes the perceived brightness of the color in the 
center in conflict with the surrounding (the darker the background the 
lighter the center will appear and vice versa). In the center the assimi- 
lation phenomenon, that changes the perceived brightness of the color 
in the center accordingly to the surrounding (the darker the back- 
ground the darker the center will appear and vice versa). Finally, on 
the bottom, the Kofka ring (Koffka, 1935), where the perceived bright- 
ness of each color changes according to the different edges. Even 
though the gray patches look different, their colorimetric values are 
always the same in every color system. This is because the difference 
in appearance is not caused by different stimuli but depends on differ- 
ent spatial arrangements of the scene. 

Outside the aperture mode and object mode configurations, color- 
imetry has not been defined yet. A nowadays open issue is clearly the 
need to develop a model that includes the entire contribution of the 
HVS mechanisms, able to objectively describe the color from different 
fields of view and spatial arrangements. 


2.3 Color recording systems 


Systems for color information recording include both color meas- 
urement instrumentation useful for uniform areas of color and image 
capture devices designed to capture spatially varying color infor- 
mation. 

Hereafter, the main technologies for color acquisition are intro- 
duced. For their technical operating principles, the readers may refer 
to the book (Sharma and Bala, 2017). 


2.3.1 Spectroradiometer and spectrophotometer 
The most direct and complete method for recording color infor- 


mation is to sample the spectral distribution. A spectroradiometer is a 
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punctual device for sample surface inspection. It measures the power 
distribution of optical radiation (SPD) or the reflectance as a function 
of wavelength. Spectroradiometers employed for color recording usu- 
ally operate in the visible region of wavelengths (ca. 360-780 nm) and 
have a spectral resolution of 1 to 2 nm. 

A spectrophotometer is a punctual tool for measuring the spectral 
reflectance of an object as well, and, unlike spectroradiometers, it does 
not measure self-luminous objects since it features an inbuilt internal 
light source that illuminates the sample under measurement. 

Both devices are useful for the color calibration of printers and 
scanners as well as for determining the color characteristics of objects. 
In the field of cultural heritage, they are usually employed for the iden- 
tification of organic and inorganic materials like dyes and pigments 
(Bruni et al., 2002; Bacci et al., 2003), especially when a database of 
frequently used pigments is available. 


2.3.2 Colorimeter and photometer 


A colorimeter is a punctual device that exploits trichromacy as 
color values in CIE XYZ, CIELAB, or other color spaces. Some col- 
orimeters have an inbuilt light source (sometimes few different illumi- 
nants are available) for the acquisition of color of reflective objects, 
whereas others measure only self-luminous or externally illuminated 
objects. This type of instrument is less expensive than spectrophotom- 
eters and spectroradiometers since it does not provide detailed spectral 
information. It is less accurate and measures only color tristimuli of- 
fering an acceptable color performance and significant speed ad- 
vantage in addition to lower cost. 

In the field of cultural heritage colorimeters are widely used in 
many applications, e.g., the study of color aging and degradation in 
relation to different parameters such as temperature, humidity (La 
Gennusa et al., 2005), illumination (Zhao et al., 2019), (Dang et al., 
2018), and/or microorganisms presence (Rosado et a/., 2019); the pro- 
duction and testing of new protective and cleaning products, which do 
not cause a chromatic alteration of the surfaces (Giorgi et al., 2002), 
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(Bartoletti et al., 2020); the color matching for restoration purposes; 
and so on. 

Photometers are single-channel devices that provide a measure- 
ment of the luminance of a self-luminous or externally illuminated ob- 
ject. It is usually employed to inspect the power spectral distribution 
of illuminant sources. 


2.3.3 Digital color cameras and color scanner 


Color recording devices such as digital color cameras and color 
scanners work on similar principles, but their intended uses are some- 
what different. Both tools capture color data by acquiring the picture 
through a series of color filters having a different spectral transmit- 
tance and sampling the colored resulting images with electronic sen- 
sors. Digital color cameras are meant to take color photos of the real- 
world scene in the same way that traditional cameras do, with the ex- 
ception that the images are captured electronically rather than on film. 
The lack of control over scene illumination is one of the issues of color 
capture that color cameras face more than scanners and colorimeters. 

Scanners are usually designed to scan images printed on paper or 
transparencies (like documents, photographs, and films), and they fea- 
ture an inbuilt light source. These devices do not need to capture the 
complete image in a single exposure like digital cameras do, thus a 
single sensor per channel is scanned across the image to enable spatial 
sampling. The use of a single sensor simplifies and improves the per- 
formance of the device, as well as allowing the use of more expensive 
and precise sensors. 


2.3.4 Multispectral and hyperspectral imaging 


Multispectral imaging is a technology that acquires an image in 
which each pixel has several channels carrying spectral information. 
Multispectral images cover a wide range of image types, from the 
common three-channel color images to hyperspectral imaging with 
hundreds of bands. Thus, the data provided by the hyperspectral 
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imaging technique are the so-called datacube or hypercube, which 
contain data represented in three dimensions: two dimensions describ- 
ing the spatial location, and a third spectral dimension representing a 
continuous spectrum (in the case of multispectral images, the third di- 
mension represents a discrete function). Thus, hyperspectral imaging 
mixes the power of spectrophotometry whit the power of imaging 
technology. It is a non-invasive and non-destructive technology that 
allows us to obtain in a single shot thousands of radiance spectra of 
the object under analysis, one for each pixel of the image. These de- 
vices are not limited to the visible range of the electromagnetic spec- 
trum, and they commonly operate from the UV to the infrared regions, 
usually ranging from 400 to 1000 nm. 

Traditionally, multiband sensors were born for remote sensing ap- 
plications, and only in the last twenty years they have been employed 
for the analysis of historical and cultural objects, e.g., for the exploi- 
tation of underdrawings in paintings (Walmsley et al., 1994); the char- 
acterization and mapping of pigments and inks in painted artifacts and 
drawings (Casini et al., 1999; Baronti, Casini and Porcinai, 1998); the 
studying of unreadable scrips revealed (Bearman and Spiro, 1996); the 
digitization purposes (Lahanier et al., 2002; MacDonald et al., 2017); 
and many others. 

For further details, the readers may refer to (Fischer and Kakoulli, 
2006). 


2.4 Digital color reproduction issues 


In order to process digital color images or 3D reconstruction, the 
object must be sampled both spatially and spectrally, acquiring their 
spectral radiance or reflectance distributions. To achieve this purpose, 
defining a standard acquisition protocol is mandatory not to run into 
wrong sampling, affected measures or bias. This is also true in the field 
of historical and cultural heritage and especially for photographic and 
audiovisual materials. Nowadays museums, libraries, and archives 
(where this type of material is usually stored) are indeed promoting 
digitization campaigns to allow wider access and fruition to the public 
to their collections. 
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Despite the existence of many regulations provided by national and 
international institutions, like the Italian ICCD (Istituto Centrale per il 
Catalogo e la Documentazione) (MiBACT ICCD, 1998) or the Euro- 
pean FADGI (FADGI, 2015) and Metamorfoze (van Dormolen, 
2012), the guidelines are usually not enough to carry out a correct and 
accurate digitization. Most of the problems, usually resolvable by sim- 
ple precautions, are rarely mentioned in the regulations. Furthermore, 
the operators that perform the digitization are often not aware of all 
the risks of wrong color reproduction and an evaluation of instruments 
performance is rarely considered, as well as an objective evaluation of 
the quality of the results. As a consequence, an uncontrolled applica- 
tion of automatic operations without an accurate knowledge of the al- 
gorithms and corrections introduced can indeed produce errors in the 
acquisition, which can lead to issues in managing data. 

Hereafter, in the next subsections we list a set of the most common 
issues that can be found along the digital color reproduction workflow. 
There are no solutions to overcome the presented problems; instead, 
the focus is on describing issues and the questions they raise. 


2.4.1 Calibration of the instruments 


One of the main open issues is the calibration of the instruments. 
Scanners and monitors are quite always not sold with adequate cali- 
bration tools, like for example the IT8 targets. Furthermore, most of 
the time calibration operations are not included in the acquisition pro- 
cess described by the national or international protocols, and all results 
in a high risk of scanning materials without any calibration. 


2.4.2 Optical veiling glare 


Another fundamental problem is instrumentation errors such as the 
transmission of frequencies, the aberration, and the formation of glare. 
These errors are caused by the presence of lenses inside the acquisition 
instruments themselves, which alter the information reaching the sen- 
sors. Optical veiling glare is indeed a light reflection-based 
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phenomenon that consists of an unwanted light-spread on the imaging 
sensors. As a result, it produces a loss of information and a decrease 
of the dynamic range of the image acquired, so that the resulting digital 
image always has a different contrast from the original one. Even 
though the glare problem is well known in the field of lens design op- 
tics, it is much less studied and considered in the imaging field. 
Measures and tests of glare in image acquisition systems started in- 
deed only some years ago (Signoroni et al., 2020). Unfortunately, 
glare is not just noise but is a systematic distortion of the acquired data. 
The values at each point of the scene are affected according to their 
spatial arrangement and magnitude, making this phenomenon scene- 
dependent and exposure time-independent. (Gianini et al., 2019) dis- 
cusses the main reasons why glare cannot be easily removed in the 
image acquisition process. 

An example of application in the field of cultural heritage is the 
assessment of the presence of glare also in hyperspectral imaging tech- 
niques (Sarti, Plutino and Rizzi, 2020). 

Glare also affects our vision since human eyes are equipped with 
lenses as well. Despite the loss of contrast in the retina can be severe, 
spatial comparisons counteract the glare phenomenon. As a result, the 
HVS is much more robust than any color recording system based on 
traditional colorimetric approach due to the spatial mechanism of vi- 
sion. Therefore, the pointwise colorimetry cannot overcome the limits 
imposed by the instruments and the problem of a correct image acqui- 
sition is still open. 

More detailed information on glare can be found in (McCann, 
Vonikakis and Rizzi, 2017). 


2.4.3 Non-uniform illumination 


Colorimetric pointwise approaches for predicting color appear- 
ances are not applicable in complex-scene with non-uniform illumina- 
tion. Unfortunately, uniform lighting distribution does not exist. Every 
point in a scene is likely to have different levels of illumination, which 
can have a big impact on the color signal and appearance. Moreover, 
while the human eye adapts to the scene, providing a visual 
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appearance that is mostly independent of the scene illumination, cam- 
eras lack these adaptation mechanisms, resulting in imaging with sig- 
nificant color casts and shifts. 

This is a very complex open issue that in the field of cultural herit- 
age, is not only concerned with digital color acquisition purposes, but 
it is also a well-known problem in the definition of an optimal illumi- 
nation for the valorization and the fruition of the works of art, espe- 
cially for museums and collections. Every exhibition needs a trade-off 
among three factors: the visitors, the cultural assets, and the architec- 
ture of the building that hosts the event (Berns and Grum, 1987; La 
Gennusa et al., 2005). First of all, the contribution of the light must be 
specifically designed for the exhibit since it can affect the experience 
of the public. Nevertheless, the illumination system must not damage 
the work of art. Because of this, the museum personnel must also con- 
sider the specific national and international regulations that define the 
condition of the range of intensities and the amount of light per year 
the historical material can tolerate — as an example see Italian regula- 
tions for museums in (MiBACT, 2000). Unfortunately, due to the na- 
ture of the buildings within which the museum is located, the lighting 
conditions cannot always be controlled. The buildings themselves can 
be considered historical monuments under the Cultural Heritage regu- 
lations. This means that they are not allowed to be changed or modi- 
fied as well as the windows cannot be obscured by curtains or shutters 
to not modify their architectural appearance. For instance, many exhi- 
bition rooms are hosted inside ancient buildings such as old castles, 
monasteries, or factories, which may have large windows that provide 
natural light to the entire room. As a result, the light variation during 
the day can significantly affect the design of the exhibition and cause 
sensitive materials to fade and degrade more quickly (Gunde, 
KraSovec and Platzer, 2005). 

In this unstable balance, the adequate assessment of the color ren- 
dering of the work of art is effectively hard to manage. This is also 
aggravated by the lack of guidelines and objective metrics related both 
to the final color perception of an object in a complex spatial arrange- 
ment and to the spatial distribution of the illumination. A light source 
is indeed characterized only by its spectral power distribution (SPD), 
which provides no information about its spatial arrangement. The 
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other important metric is the Color Rendering Index (CRI) (CIE, 
1987), which tries to objectively define the quality of a light source by 
comparing its SPD with an ideal reference illuminant. Even though 
different rendering indexes have been introduced during the years, 
CRI still presents some criticism, especially for lighting systems with 
spiky emission spectra such as LEDs. The computed colorimetric 
value of CRI is indeed not always in line with the perceived final color 
evaluation. As a result, the chromatic rendering decision is usually left 
to the subjectivity of the curator of the museum or the light designer. 


2.4.4 Complex real scene 


Several parameters can affect the final color perception of a com- 
plex scene, such as the illuminant spectral and spatial distribution, the 
geometry of illumination, the observer’s point of view, and, above all, 
the shape and the size of the objects. The non-correspondence between 
computed and perceived color can be observed using different color 
recording systems. Data acquisition tasks can indeed have some criti- 
cisms like the inappropriate measurement conditions. As already said, 
the colorimetric approaches of aperture mode and object mode, con- 
strain the acquisitions to precise standard parameters of geometry of 
illumination, direction of observation and light source. Outside these 
conditions, traditional colorimetry will usually fail. This is the case of 
complex 3D real scenes. 

For instance, an application that remarks the limits of pointwise 
colorimetric measurements is the color evaluation of gemstones (De 
Meo, Plutino and Rizzi, 2020). Even though their color assessment is 
a key factor for determining the market price of a gem, defined stand- 
ards and guidelines for their color evaluation still have not been intro- 
duced yet. Furthermore, the lack of a commonly shared protocol, the 
refractive properties of these materials make the task even harder be- 
cause of the extremely complex measure condition. The objective col- 
orimetric measures have indeed provided different results compared 
to the subjective human eye evaluation, which instead reached the 
highest precisions. As a result, the color assessment of gemstones is 
still visually performed by experts. 
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Other applications in which the human eye evaluation still provides 
better results are, for instance, the color assessment of materials under 
LEDs illumination; the hair color evaluation; the variation assessment 
of color after restoration work. In all these cases pointwise approach 
fails because it tries to compute a color evaluation of a complex con- 
test without considering the spatial processing of vision. 


2.4.5 Bidimensional representation of real scenes 


All the problems mentioned above are valid not only for image or 
film digitization but also for 3D reconstruction purposes. The creation 
of the 3D digital model consists of acquiring images or scans of the 
object under analysis from various points of view and then merging 
the information into a single digital model consisting of a dense cloud 
of points or a set of triangles. Thus, 3D acquisition consists of the cre- 
ation of a three-dimensional digital model that faithfully represents the 
shape and color characteristics of an object. In practice, the 3D digital 
model is an accurate description of the surface of the object under ex- 
amination. Since the starting point of this technique are still images, 
the targets and tools available for calibration, contrast correction, and 
color management are the same. However, there is also another im- 
portant complexity to deal with: when acquiring an image, a 3-D ob- 
ject scene is projected into a 2-D representation, but a bidimensional 
representation of a real scene is not the same as looking directly at the 
3-D object (Rizzi, 2021b). 


2.4.6 Color management 


Once images are acquired, colors are a fundamental parameter dur- 
ing data processing since they influence the results of the algorithms. 
Usually, a color image processing pipeline follow the following steps: 
(a) exposure estimation, (b) pre-processing (e.g., noise removal), (c) 
linearisation, (d) dark current compensation, (e) flare compensation, 
(f) white balance, (g) demosaicing, (h) color transformation (in 
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unrendered and rendered color spaces), (1) post-processing (Plutino 
and Simone, 2021; Ramanath et al., 2005). 

While the pipeline structure is generally standardized, each step can 
be carried out using a variety of algorithms, metrics, and measure- 
ments based on several reference targets, like the ColorChecker, the 
SFR, or the IT8. In addition, the quality of several features, such as 
color reproduction, noise reduction and edge preservation, can be as- 
sessed by the system performance evaluation. As a result, each pipe- 
line is unique for its specific application field. 

The ColorChecker is one of the most common targets for image 
correction in image processing and photogrammetric pipelines. This 
chart, which was first introduced in 1976, is nowadays employed in 
photography, scanning, and object acquisition to check color stability. 
However, several color reproduction errors may occur, and several 
studies proved the ColorChecker to be insufficient to ensure color re- 
production robustness. As a result, the entire photogrammetric pro- 
cess, as well as object digital reconstruction, may be biased and not 
faithful (Gaiani et al., 2017). 

Cultural heritage digitization differs from other imaging applica- 
tions because aiming at future repurposing and durability. Therefore, 
in this domain it is crucial not to lose information, and the digitization 
process must provide digital images perceptually faithful to the origi- 
nal object, rather than pleasant. As a result, contrast and color repro- 
duction cannot be approximated, and inaccuracies must be kept to a 
minimum and constantly monitored. Several image quality assessment 
methods and many difference/quality measures have been introduced 
to achieve a faithful digital image, but they are still dependent on 
pointwise colorimetry (Barricelli et al., 2020). As already underlined, 
pointwise-based colorimetry has been proven to be still unable to re- 
produce colors accurately and fails in complex scenes with non-stand- 
ard expositions and with different light sources. The digital acquisition 
system and technologies at current reading are still not sufficient to 
guarantee a faithful reproduction of the historical and cultural assets, 
thus new solutions are required. 
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2.5 Conclusions 


In this chapter we have underlined and recalled the assumptions and 
constraints beyond colorimetry, which are often not considered or 
even not truly understood. Their description can help in comprehend- 
ing possible mismatches between computed and perceived color sen- 
sations in many applications. 

The description of the current analytical technologies employed to 
analyze color information was meant to show the wide applications in 
which colorimetry can support the study of historical and cultural ob- 
ject, as well as to facilitate the explanation of the most prevalent and 
challenging issues among all the color digitization pipeline. 

Color analysis and reproduction work fine in aperture mode, but 
still fail outside pointwise colorimetry constraints and are often sub- 
stituted with subjective assessment. There is the need to develop new 
methods for computing color mechanisms in complex scenes, that is, 
considering spatial interactions, and not just input signal of cones. 


References 


Bacci, M. et al. (2003) ‘Non-invasive spectroscopic measurements on the Il ritratto 
della figliastra by Giovanni Fattori: identification of pigments and colourimetric 
analysis’, Journal of Cultural Heritage, 4(4), pp. 329-336. doi: 
10.1016/j.culher.2003.09.003. 

Baronti, S., Casini, A. and Porcinai, S. (1998) ‘OSA | Multispectral imaging system 
for the mapping of pigments in works of art by use of principal-component anal- 
ysis’, Applied Optics, 37(8), pp. 1299-1309. doi: 10.1364/A0.37.001299. 

Barricelli, B. R. et al. (2020) ‘A cockpit of multiple measures for assessing film 
restoration quality’, Pattern Recognition Letters, 131, pp. 178-184. doi: 
10.1016/j.patrec.2020.01.009. 

Bartoletti, A. et al. (2020) ‘Reviving WHAAM! a comparative evaluation of clean- 
ing systems for the conservation treatment of Roy Lichtenstein’s iconic paint- 
ing’, Heritage Science, 8(1), p. 9. doi: 10.1186/s40494-020-0350-2. 

Bearman, G. H. and Spiro, S. I. (1996) ‘Archaeological Applications of Advanced 
Imaging Techniques’, The Biblical Archaeologist, 59(1), pp. 56-66. 
doi: 10.2307/3210537. 

Berns, R. S. and Grum, F. (1987) ‘Exhibiting artwork: Consider the illuminating 
source’, Color Research & Application, 12(2), pp. 63-72. doi: 
10.1002/c01.5080120204. 


44 


Copyright © 2021 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788835125716 


Bruni, S. et al. (2002) ‘Field and Laboratory Spectroscopic Methods for the Identi- 
fication of Pigments in a Northern Italian Eleventh Century Fresco Cycle’, Ap- 
plied Spectroscopy, 56(7), pp. 827-833. doi: 10.1366/000370202760171482. 

Casini, A. et al. (1999) ‘Image spectroscopy mapping technique for noninvasive 
analysis of paintings’, Studies in Conservation, 44(1), pp. 39-48. doi: 
10.1179/sic.1999.44.1.39. 

CIE (1987) ‘TEC/CIE 017.4-1987 - International lighting vocabulary’. Available at: 
https://www.techstreet.com/standards/iec-cie-017-4-1987?product_id=47660 
(Accessed: June 2021). 

Dang, R. et al. (2018) ‘Chromaticity changes of inorganic pigments in Chinese tra- 
ditional paintings due to the illumination of frequently-used light sources in mu- 
seum’, Color Research & Application, 43(4), pp. 596-605. doi: 
10.1002/col.22215. 

De Meo, S., Plutino, A. and Rizzi, A. (2020) ‘Assessing color of gemstones’, Color 
Research & Application, 45(2), pp. 224-234. doi: 10.1002/col.22472. 

Dormolen van, H. (2012) ‘Metamorfoze Preservation Imaging Guidelines’. Meta- 
morfoze - National programme for the preservation of paper heritage. Available 
at: https://www.metamorfoze.nl/sites/default/files/publicatie_documenten/Met- 
amorfoze_ Preservation Imaging Guidelines 1.0.pdf (Accessed: June 2021). 

FADGI (2015) ‘Technical Guidelines for Digitizing Cultural Heritage Materials’. 
Federal Agencies Digital Guidelines Initiative. Available at: http://www.digiti- 
zationguidelines. gov/guidelines/digitize-technical.html (Accessed: June 2021). 

Fischer, C. and Kakoulli, I. (2006) ‘Multispectral and hyperspectral imaging tech- 
nologies in conservation: current research and potential applications’, Studies in 
Conservation, 51(sup1), pp. 3-16. doi: 10.1179/sic.2006.51.Supplement- 1.3. 

Gaiani, M. et al. (2017) ‘Securing Color Fidelity in 3D Architectural Heritage Sce- 
narios’, Sensors, 17(11), p. 2437. doi: 10.3390/s17112437. 

Gianini, G. et al. (2019) ‘Glare removal as an ill-conditioned problem’, Journal of 
Electronic Imaging, 28(6), doi: 10.1117/1.JEI.28.6.063014. 

Giorgi, R. et al. (2002) ‘Nanotechnologies for Conservation of Cultural Heritage: 
Paper and Canvas Deacidification’, Langmuir, 18(21), pp. 8198-8203. doi: 
10.1021/la025964d. 

Gunde, M. K., KraSovec, U. O. and Platzer, W. J. (2005) ‘Color rendering properties 
of interior lighting influenced by a switchable window’, JOSA A, 22(3), pp. 416- 
423. doi: 10.1364/JOSAA.22.0004 16. 

Koffka, K. (1935) Principles of Gestalt Psychology. London: Lund Humphries. 

La Gennusa, M. et al. (2005) ‘Control of indoor environments in heritage buildings: 
experimental measurements in an old Italian museum and proposal of a method- 
ology’, Journal of Cultural Heritage, 6(2), pp. 147-155. doi: 
10.1016/j.culher.2005.03.001. 

Lahanier, C. et al. (2002) ‘CRISATEL: high definition spectral digital imaging of 
paintings with simulation of varnish removal’, ICOM Committee for Conserva- 
tion, ICOM-CC: 13th Triennial Meeting, Rio de Janeiro, 22-27 September 2002: 
preprints, pp. 295-300. 


45 


Copyright © 2021 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788835125716 


MacDonald, L. W. et al. (2017) ‘Assessment of multispectral and hyperspectral im- 
aging systems for digitisation of a Russian icon’, Heritage Science, 5(1), p. 41. 
doi: 10.1186/s40494-017-0154-1. 

McCann, J. J., Vonikakis, V. and Rizzi, A. (2017) HDR Scene Capture and Appear- 
ance. SPIE. doi: 10.117/3.2315540. 

MiBACT (2000) ‘Atto di indirizzo sui criteri tecnico-scientifici e sugli standard di 
funzionamento e sviluppo dei musei (D. Lgs. n.112/98 art. 150 comma 6)’. 
Available at: http://musei.beniculturali.it/wp-content/uploads/2016/04/Atto-di- 
indirizzo-sui-criteri-tecnico-scientifici-e-sugli-standard-di-funzionamento-e- 
sviluppo-dei-musei-DM-10-maggio-2001.pdf (Accessed: June 2021). 

MiBACT ICCD (1998) ‘Normativa per l’acquisizione digitale delle immagini foto- 
grafiche’. Ministero per i beni e le attività culturali, Istituto centrale per il cata- 
logo e la documentazione. Available at: http://\www.iccd.benicul- 
turali.it/it/152/pubblicazioni-1ccd/4182/normativa-per-l-acquisizione-digitale- 
delle-immagini-fotografiche (Accessed: June 2021). 

Oleari, C. (2015) Standard Colorimetry: Definitions, Algorithms and Software. 
Chichester, UK: John Wiley & Sons. 

Plutino, A. and Simone, G. (2021) ‘The limits of colorimetry in cultural heritage 
applications’, Coloration Technology, 137(1), pp. 56-63. doi: 
10.1111/cote.12500. 

Ramanath, R. et al. (2005) ‘Color image processing pipeline’, IEEE Signal Pro- 
cessing Magazine, 22(1), pp. 34-43. doi: 10.1109/MSP.2005.1407713. 

Rizzi, A. (202 1a) ‘Colour after colorimetry’, Coloration Technology, 137(1), pp. 22- 
28. doi: 10.1111/cote.12496. 

Rizzi, A. (2021b) ‘What if colorimetry does not work’, In press. 

Rosado, T. et al. (2019) ‘Understanding the influence of microbial contamination on 
colour alteration of pigments used in wall paintings—The case of red and yellow 
ochres and ultramarine blue’, Color Research & Application, 44(5), pp. 783-789. 
doi: 10.1002/col.22391. 

Sarti, B., Plutino, A. and Rizzi, A. (2020) ‘Glare ottico nelle immagini iperspettrali’, 
XVI Conferenza del colore. 

Sharma, G. and Bala, R. (2017) Digital Color Imaging Handbook. CRC Press. 
Signoroni, A. et al. (2020) ‘Spatial-Spectral Evidence of Glare Influence on Hyper- 
spectral Acquisitions’, Sensors, 20(4374), pp. 1-18. doi: 10.3390/s20164374. 
Walmsley, E. et al. (1994) ‘Improved visualization of underdrawings with solid-state 
detectors operating in the infrared’, Studies in Conservation, 39(4), pp. 217-231. 

doi: 10.1179/sic.1994.39.4.217. 

Zhao, Y. et al. (2019) ‘Degradation of red lead pigment in the oil painting during 
UV aging’, Color Research & Application, 44(5), pp. 790-797. doi: 
10.1002/col.22386. 


46 


Copyright © 2021 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788835125716 


3. Strengthening knowledge of the transition to a 
circular economy in the furniture sector 


James Christopher Postell, Mauro Attilio Ceconello 
Department of Design, Politecnico di Milano 


Abstract 


This chapter focuses on thematic areas of the circular economy, re- 
flecting the goals of the European Green Deal and addresses gains 
made by European furniture companies. The goal is to explore guide- 
lines and assessment criteria for how the design and manufacturing of 
furniture can contribute to a climate neutral Europe by 2050 with a 
sustainable economy that leaves no one behind. How are European 
furniture companies adapting to a circular economy model? 

A circular economy aims to preserve the value of products, compo- 
nents, and materials within a larger economic system. It is oriented to 
achieve more efficient production and consumption systems, thanks to 
continuous and regenerative cycles. It also focuses on reduction of raw 
materials and energy consumption, waste, and emissions generation in 
the production processes. The adoption of the circular economy re- 
quires a change of business, territorial, and individual visions, and to 
rethink ways of producing and consuming. 

The authors study evidence of the circular economy values, such as 
reuse, recycle, remanufacturing in the European furniture sector. By 
developing the knowledge base for greener and smarter manufacturing 
processes, it is possible to plan for a sustainable and more democratic 
European design culture. 
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3.1 Introduction 


This chapter focuses on circular economy (CE) strategies in the fur- 
niture sector and offers examples where furniture companies in Europe 
are responding to the transition. We explore the values of the CE 
model and identify strategies for European furniture companies to shift 
to a more socially democratic, healthier, and sustainable future. Which 
companies are leaders and innovators of CE solutions and strategies? 
By strengthening the knowledge base for greener solutions and CE 
manufacturing processes, it is possible to foresee a sustainable and 
more democratic European design culture achieve the goals outlined 
in the European Green Deal. 

Furniture relies on materials and energy for its manufacturing and 
distribution to consumers throughout the world. Across the sector, 
manufacturing utilizes a hybrid of analog and smart digital technolo- 
gies that result in making innovative solutions, systems, and services. 
The origin of the word manufacturing connects production to a pro- 
cess of making with one’s hands using hand tools. The term has Latin 
origin, a noun, denoting something made by hand; from French, re- 
formed by association with Latin manu factura ‘made by hand’ 
(Etymonline, 2021). In the past 50 years, there has been an accelerated 
transition towards technological innovation in how most furniture is 
made, transitioning away from hands to machines, and more recently, 
to CAM technologies and digitally programmable robots. How things 
are made is the basis of manufacturing, but now, more than ever, man- 
ufacturing needs to be reconsidered in tandem with an important envi- 
ronmental challenge: Will the world be made a better place to live in 
tomorrow, by how things are made today? 

We need to consider sustainable practices in using materials and 
smart processes that consume less energy in the resource extraction, 
transport, manufacture, and distribution of furniture products. Manu- 
facturing has a profound impact on the health and welfare of those who 
make and use furniture, but also impacts the planet and environment 
regarding the energy consumed in the extraction, use, and reuse of ma- 
terials and products. We begin by introducing environmental and eco- 
nomic challenges and an innovative program titled the European 
Green Deal. This is followed by a discussion of what is a circular 
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economy. We then assess the global furniture industry and highlight 
European furniture companies adapting to this changing direction. 


3.2 Background to the European Green Deal 


The world today is facing complex environmental, societal, and 
economic challenges. The intersectionality of these challenges: cli- 
mate change, environmental concerns of sustainability, health, and 
welfare issues, growing populations and urbanization, financial crises, 
shifts in societal lifestyle and values, and the rise of smart and digital 
IoT technologies, all contribute to the need to pause and reflect on the 
influence that manufacturing solutions, systems, and strategies can 
have on our lives. Manufacturing depends on designers and companies 
who address these realities and relies on consumer’s buy-in to gauge 
successful outcomes. Smart strategies can serve as a foundation for 
innovative manufacturing strategies and solutions that benefit society, 
regional and global economies, and the environment. There is a need 
to develop further guidelines, and utilize assessment criteria for CE 
manufacturing, specifically by leading European furniture companies 
to help achieve climate neutrality with a sustainable economy by 2050. 
The European Union has begun that process. 

On 11 December 2019, the EU Commission announced the Euro- 
pean Green Deal (EU, European Commission, Secretariat-General, 
2019) to transform the EU into the first climate-neutral continent by 
2050. The plan is to improve the EU’s economy, by turning climate 
and environmental challenges into opportunities. It is a growth-strat- 
egy to transform the EU into a more sustainable, more environmen- 
tally respectful place of production and consumption by adopting the 
CE model. The Green Deal’s goals should result in a competitive cir- 
cular economy, where there are no net emissions greenhouse gases by 
2050, economic growth is decoupled from resource use, and no place 
is left behind. 

Following the creation of the European Innovation Council (EIC), 
the European Commission launched a €1 billion call for research and 
innovation projects that: respond to the climate crisis; support the Eu- 
ropean Green Deal initiative; and help protect Europe’s unique 
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ecosystems and biodiversity. These initiatives were conceived before 
the COVID-19 pandemic. The Horizon 2020-funded European Green 
Deal call was created to spur Europe’s recovery from the COVID-19 
pandemic by stimulating transformative “green” challenges into inno- 
vation, opportunities, and solutions. 


3.3 Overview of the Circular Economy 


The CE model is gaining interest as a new industrial paradigm 
which demonstrates a smarter alternative to the linear economy 
(Murray, Skene and Haynes, 2017). The primary goal of a CE is to 
decouple economic growth from natural and limited resource con- 
straints and connect the benefits of multi-use to proven sustainable 
practices. Therefore, the definition of the CE that best explains this 
research is: 


A sustainable development initiative with the objective of reducing the societal 
production-consumption systems linear material and energy throughout flows by ap- 
plying material cycles, renewable and cascade-type energy flows to the linear system 
(Korhonen et al., 2018). 


The CE model is a smart approach to design, production, and con- 
sumption. It contrasts with the existing model of production and con- 
sumption identified as a linear economy. The linear economy is a pro- 
cess of making, using, and discarding products. It has been the para- 
digm for most worldwide industrial output, including the manufactur- 
ing of furniture. CE is an industrial system of manufacturing, distribu- 
tion, and consumption that is restorative and regenerative by design. 
The term restorative suggests a closed-loop cycle that encourages 
more output and utility from products than what is delivered in a linear 
economy. In a restorative system, technical materials and products are 
recovered and given new life. In a regenerative system, biological nu- 
trients are reproductive (Ellen MacArthur Foundation, SUN and 
McKinsey, 2015). 

CE is not a new concept, but it has taken time to develop a concep- 
tual framework with obtainable goals. A goal of CE is to extend the 
life and value of furniture through innovative manufacturing processes 
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enabling reuse, repair, refurbishment, and remanufacturing cycles in 
production strategies that respond to environmental challenges out- 
lined in the European Green Deal (2019). 

In striving for a holistic and theoretical understanding of circular 
economy implementation, the authors leverage some of the conse- 
quential literature produced thus far (Simmonds, 1862; Ayres and 
Kneese, 1969; Kenneth E. Boulding, 1966; McDonough and 
Braungart, 2003; Lewis and Slack, 2014; Unal, Urbinati and Chiaroni, 
2019) and the CE research, publications, and video presentations exe- 
cuted by the Ellen MacArthur Foundation (Ellen MacArthur 
Foundation, 2013). This collective research has helped frame options 
available to transform many industrial practices currently dependent 
on exploitation. 

The novelty of CE relies upon a system where products are de- 
signed and manufactured to be used longer, applying any number of 
“value cycles” or closed-loop strategies to extend the value of the 
product beyond its conventional end-of-life (EoL) use (Ellen MacAr- 
thur Foundation, 2015). CE is a change in thinking in the approach to 
how we think about making things. It is a concept that puts [re] in front 
of use, make, and manufacture, placing value on innovative ways to 
re-use things. It is valuable and complementary to the Cradle-to-Cra- 
dle (C2C) concept (McDonough and Braungart, 2003). C2C has influ- 
enced thinking about values in design and manufacturing and has set 
the stage for today’s discussion on the CE. 


Our goal is a delightfully diverse, safe, healthy, and just world, with clean air, 
water, economically, equitably, ecologically and elegantly enjoyed (McDonough, 
2007). 


This statement presents a new connection to why we design and 
make things. 

The Ellen MacArthur Foundation developed a diagram that illus- 
trates two intersecting economic systems, (liner and circular). Figure 
1 illustrates two closed loop cycles located in distinct sectors: one bi- 
ological and the other technical. The central vertical axis represents 
the linear economic model, while the rest of the diagram illustrates the 
continuous flow of materials and processes referred to as value cycles. 
The furniture sector is concerned with technical cycles. Including both 
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biological and technical systems remind us that the furniture sector can 
draw values from both. Any one of these value cycles can extend the 
life of a product. A combination of value cycles manifests a traceable 
impact on the economy and the environment. These two models are 
often defined as cradle-to-grave (linear economy) versus cradle-to- 
cradle (circular economy) (Ellen MacArthur Foundation, SUN and 
McKinsey, 2015). 
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Fig. 3.1 - The Butterfly Diagram for Circular Economy. Circular economy systems 
diagram. Ellen MacArthur Foundation (2019). Drawing based on Braungart & 
McDonough, Cradle to Cradle. Copyright © Ellen MacArthur Foundation (2019), 
www.ellenmacarthurfoundation.org. 


Addressing the critical problem of natural global resource deple- 
tion, the Ellen MacArthur Foundation (2013) has summarized four 
principles of the CE as “points ofaction” to revitalize existing material 
value throughout one or more stages in the manufacturing of products. 
The authors see value in the four principles and will later synchronize 
them to exemplary cases. It is important to note these are complemen- 
tary, yet distinct from green and sustainable principles. Subcategories 
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(*) have been added by the authors to the four points of action related 
to the furniture sector: 


1. Optimize the use of resources and energy throughout all life cycle 
stages. 

* Design for efficiency in the use of material, time, and energy. 

2. Maintain production and components in use over a longer time. 

* Improve furniture design by thinking about modularity, standard- 
ization, and compatibility with existing components and assem- 
blies (open platform). 

* Generate personal attachment to furniture. 

* Manufacture for self-maintenance and repair. 

* Consider ease of disassembly and reassembly. 

3. Cycle materials through the production system as many times as 
possible. 

* Create opportunities for upgrading downgrading, adaptability, re- 
use and repurpose. 

* Consider the recyclability of material. 

4. Utilize pure materials to improve quality of post-life use. 

* Use materials that maximize reliability and durability. 


CE achieves more efficient production and consumption outcomes 
by moving products and components through continuous use cycles so 
that at the product’s EoL there is minimum residual of a product that 
ends up as waste or in a landfill (Rios and Charnley, 2017). CE focuses 
on ways to re-cycle a product through stages of production to exploit 
all its reiterations of use. The full adoption of the circular economy 
within an industry requires a change in how society thinks about value 
and accepts new ways of consuming products through various cycles. 

Cyclical, closed-loop processes allow for a broad range of strate- 
gies and actions for furniture such as: 


e Refurbishing: remanufacturing a product to optimize its life. 

e Restoring: refinishing or re-upholstering to extend the condition of 
a product. 

e Repairing: corrective repair of a product. 

e Maintaining: preventative maintenance to maximize product life 
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e Reusing: redistributing products through a change in ownership. 

e Repurposing: changing the functionality of a product, (i.e., a chair 
becomes a table). 

e Recycling: recovering the value of materials and components in 
products for reuse. 

e Regenerative: a process of regrowing (renewing). 


A CE is suitable for technological industries to adopt because it 
opens avenues for smart and innovative ways of using closed-loop 
processes. Also important is the reduction of raw materials and energy 
consumption, waste, and emissions generation in the production pro- 
cesses. Adopting the circular economy model does require a change in 
business practices and consumer behaviors. CE requires everyone to 
rethink the way products and services are produced and consumed. We 
examine the CE paradigm by applying the concept to the furniture sec- 
tor to examine how well a CE can change an existing linear economy. 


3.4 Overview of the Global and European Furniture Sectors 


The furniture industry today accounts for a considerable portion of 
regional and global trade with approximately one million workers em- 
ployed in the EU (EEB, 2017). The global furniture market was esti- 
mated at approximately €285 billion in 2019 and is expected to reach 
approximately €390 billion by 2026. Approximately one-quarter of 
the world’s furniture manufacturing occurs within the EU28 member 
states, representing a €84 billion market that equates to a EU28 con- 
sumption of about 10.5 million tons of furniture per year (EU 
FURN360, 2017). 

So often the case, after furniture no longer serves its primary pur- 
pose, if not sold to another user or repurposed for another use, it is 
discarded for any number of reasons into a landfill. According to the 
European Environment Bureau (EEB), 10 million tons of furniture are 
discarded by businesses and households as waste every year (EEB, 
2017). This accounts for over 4% of the total Municipal Solid Waste 
(MSW) stream in the EU. Sadly, 80 to 90% of EU furniture waste in 
MSW was incinerated or sent to landfill with only 10% recycled (EU 
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FURN360, 2017). Waste arising from commercial sources is assumed 
to contribute 18% of total furniture waste generation across the sector. 
These numbers represent a cradle-to-grave life cycle: an unsustainable 
ecosystem. 

In 2017, most of the EU furniture companies in the sector were 
small and medium-sized enterprises (SMEs). Italy (€17.5 billion), 
Germany (€14.5 billion), and UK (€8.8 billion) were the most signifi- 
cant furniture producers by value (EEB, 2017). The most significant 
exporters were Germany (€9.5 billion), Italy (€9.2 billion) and Poland 
(€8.7 billion). The largest importers were Germany (€11.8 billion), 
UK (€6.6 billion) and France (€6.0 billion). European member states 
are major consumers of furniture with a considerable proportion from 
wooden furniture, kitchen units, and mattresses. 

The statistics on EOL production and waste are not limited to the 
EU. Historically, the furniture sector has utilized a traditional business 
model that follows a linear process: one of harvesting materials, mak- 
ing, using, then discarding products as waste. Waste is often burned or 
ends up in landfills, characteristic of the cradle-to-grave model. Addi- 
tionally, this sector is known to use virgin raw materials (wood, stone, 
textiles) for production and uses adhesives, dyes, and coating materi- 
als to finish and protect furniture, which can result in emission of vol- 
atile organic compounds (VOCs). Regardless of the material con- 
sumed and technology used in the manufacturing of furniture, compa- 
nies that focus on CE methods are reimagining the value of furniture 
by incorporating closed-loop cycles. We now explore tools and re- 
sources that provide opportunity and evidence of transition from a lin- 
ear to the circular economic model. 


3.5 CE Resources and Tools Available for the Furniture 
Sector 


Resources and tools are available to advance sustainable furniture 
design, serving to build adoptable strategies and strengthen knowledge 
in the sector. Annotated projects also help benchmark sustainability. 
One project was coordinated by Centro Legno Arredo Cantù (CLAC) 
Ecomind (Centro Legno Arredo, Material Connexion and Istituto 
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Europeo di Design, 2006). It was the result of collaborative work that 
highlights ecological furniture solutions and material strategies. A sec- 
ond is Eco-Design (Fuad-Luke, 2010), cataloging a broad range of 
product and furniture solutions designed for a sustainable future. Both 
sources strengthen knowledge about sustainable and eco-sensitive 
strategies that benefit the environment and argue that eco-sustainabil- 
ity criteria should receive the same weight as technical, functional, 
aesthetic, ergonomic and economic considerations in the design pro- 
cess. Surveys and assessments serve as measurements of practice and 
attitudes that assists the adoption of a sustainable, methodological, de- 
sign approach for innovative solutions in the various stages of the 
product life cycle is the adoption of the Life Cycle Assessment (LCA) 
(Ceschin and Gaziulusoy, 2016). The Ecodesign Directive by the Eu- 
ropean Commission (EU, 2005) provides directions for companies to 
use LCA to evaluate their productive processes and products. This 
self-evaluation tool helps identify CE practices being used and sug- 
gests others to incorporate in practice. This is a CE approach that many 
companies in the wood furniture sector can adopt to raise quality 
standards and be more competitive in a market that is increasingly at- 
tentive to the use of energy, materials, and certifications. The link be- 
tween sustainability and innovation, in fact, is becoming increasingly 
intertwined. 

A report from Federlegno Arredo (FederlegnoArredo, Symbola, 
2016) highlights latest trends in the production of the furniture sector 
among leaders in the European market. It reports on low energy con- 
sumption, reduction of emissions, waste reduction, and recycling of 
materials. Furniture enterprises are identified by their green practices. 

In recent years legislation at the European and international level 
has defined rules for the certification of raw materials from responsi- 
bly managed and environmentally sustainable sources. Among the 
most important are the Programme for Endorsement of Forest Certifi- 
cation (PEFC) schemes and the Forest Stewardship Council (FSC). 
These programs guarantee that the raw material used in the manufac- 
ture of furniture comes from controlled cultivation of plants, guaran- 
tees protocols that avoid deforestation, (the consequent increase of 
CO2), and commits to maintain biodiversity in the areas of origin. Ac- 
cording to the directives of the European Regulation 995/2010 (EU, 
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2010), materials are verified and certified, defining their origin and 
adherence to the standards provided. Furniture manufacturers and de- 
signers who source certified raw materials can add the endorsement 
programs to their marketing materials. 

From the circular economy perspective, designers can use an eco- 
sustainable approach to design using strategies that consider varied 
factors in product development (Pigosso, McAloone and Rozenfeld, 
2015). Not only by focusing on the formal and material characteristics 
of a sustainable product, but also considering the link between the 
product to other product processes such as packaging, distribution, 
marketing, and social performance. This business solution at the de- 
sign phase can impact waste issues in localized markets. Chiu and Kre- 
mer (2011) provide an exhaustive analysis of Design for X guidelines 
and a toolkit, where X stands for any attitude in the design process 
(Manufacturing, Assembly, Disassembly, Logistics, ...) to supply 
practitioners with an index for each DfX concept and method. In sum- 
mary, a variety of resources and tools have been developed to 
strengthen CE knowledge in the furniture sector. Undoubtedly, there 
will be more to come. The next section explores the Points of Action 
outlined in the Overview of CE section and synchronizes them to in- 
novative case studies. 


3.6 Case studies and practices: innovative CE solutions 
and strategies 


There is growing evidence of furniture designers and companies 
adopting CE values to improve furniture design and manufacturing 
processes. In this section, we provide examples which align with the 
previously identified four points of action that were discussed in 3.3 
Overview of the Circular Economy. These furniture solutions and 
manufacturing companies exhibit substantial effort to commit to the 
CE value system. Following are six examples of innovative CE solu- 
tions and strategies that represent changes the furniture industry is 
making to extend material value and reduce consumption of energy 
throughout one or more stages in the manufacturing of furniture prod- 
ucts. 
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3.6.1 Optimize the use of resources and energy throughout 
all life cycle stages 


Companies in the furniture sector are seeking opportunities to opti- 
mize the use of resources and energy. The examples that follow were 
selected because they demonstrate forward thinking solutions for the 
management of resources and energy in manufacturing and distribu- 
tion. 


Fig. 3.2 - The Plus has been designed by BIG - Bjarke Ingels Group (2021) based 
on sustainable and CE values. The Plus Aerial Image credit: Lucian R. 


Vestre is a Norwegian furniture manufacturer that commits to long 
term sustainability, a prerequisite in all parts of their operations, their 
design of furniture solutions, procurement of materials, and their man- 
ufacturing to distribution processes. Using innovative robotic technol- 
ogy, their new manufacturing plant “The Plus” (designed by BIG - 
Bjarke Ingels Group and scheduled for completion in Autumn 2021), 
will be carbon zero. The new factory will be the first industrial build- 
ing in the Nordic region to achieve the highest environmental certifi- 
cation, BREEAM Outstanding, by using solid wood and ensuring a 
fossil-free and emission-free construction site. The Plus will employ 
several Industry 4.0 solutions, such as smart robots and self-driving 
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trucks, and the whole factory can be run from a tablet device. Every 
aspect of the factory will be based on principles of renewable and clean 
energy to ensure eco-friendly production. The Plus will produce 50% 
lower greenhouse gas emissions than similar conventional factories. 

Additive manufacturing is a straightforward process of making that 
can impact the production of furniture components from a three-di- 
mensional model adding material layer by layer in filament, liquid, or 
powder through a digital-driven technological process that can replace 
or complement traditional fabrication methods. Fields of applications 
in additive manufacturing, range from prototyping to actual produc- 
tion, and given the widespread use of this tool, an assessment of the 
environmental dimensions concerning the production phase and the 
entire life cycle of the components produced is needed (Kellens et al., 
2017). 

An example of AM utilized in the furniture sector is The Endless 
Chair by Dirk van der Mooij (2011). It was designed using CAD tech- 
nologies and digital scripts and manufactured using robotic CAM 
tools. AM processes are efficient, and waste is minimized. The End- 
less Chair illustrates the potential in digital CAD / CAM technology 
to transform the furniture sector. 


Fig. 3.3 - The Endless Chair, manufactured utilizing AM technologies. Dirk van 
der Kooij for Studio Kooij (2011). 


59 


Copyright © 2021 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788835125716 


High-technological processes are pathways toward achieving sus- 
tainability goals and adopting the circular economy model’s values. 
Research suggests that this path can become more circular and needs 
further study. 


3.6.2 Maintain production and components in use over a 
longer time 


The concept of maintaining production of components is critically 
important in extending the life cycle cost (LCC) of manufactured fur- 
niture. Here are a couple of examples demonstrating practices that al- 
ready extend the life of furniture through the manufacture of modular 
and standardized components and the accessibility to the components 
by consumers. 

IKEA has a global presence with global markets and distribution 
aligned with CE principles. Much of their furniture embodies modu- 
larity and standardization of components which allows for replace- 
ment and repair of its furniture products. IKEA created a material re- 
covery program for home furniture, a take-back campaign for recy- 
cling furniture and is pursuing the use of waste from production as a 
resource for new products which reduces landfills. The company 
places significant value on democratic design, a concept of making 
clever design available to everyone. IKEA expands the concept of 
democratic design into five dimensions, which are: function, form, 
quality, sustainability, and low price. When there is a balance between 
all five dimensions, IKEA considers the design “democratic” 
(IKEA.com, 2021). 

Re-manufacturing is a feasible alternative to reusing furniture 
where processes of checking, resurfacing and redeployment apply to 
the long-life components and assemblies, like steel frames or struc- 
tural parts. Rype Office, a London-based company, remanufactures 
brand name office furniture back to as-new condition and creates sus- 
tainable furniture from potential waste materials. The result looks and 
performs as new, but at a significant cost savings, economically and 
environmentally. Remanufacturing is a strategy that commits to a 
zero-landfill goal by repurposing old furniture for other uses and/or 
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recycling them into new usable products. Key sustainable factors in 
the process are eco-sustainability, reducing raw resources, energy, and 
water, and being in position to sell functioning furniture at a reduced 
price. 


3.6.3 Cycle materials through the production system as 
many times as possible 


The furniture sector is a potential leader for the action of cycling 
materials through the manufacturing process as many times as possi- 
ble. Following are examples of how recycling and upcycling can be 
used to reduce single-use materials. 


Fig. 3.4 - Recycled Grey Babila XL by Odo Fioravanti, Pedrali (2020). 


Pedrali, an Italian furniture company active in sustainable design, 
aims to design and manufacture products able to last through time for 
style and performance. Their collection titled “Recycled Grey” is 
made of 100% recycled plastic composed of 50% post-consumer and 
50% industrial plastic waste. Post-consumer plastic waste comes from 
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products previously used by consumers, such as plastic bottles or food 
packaging, while the industrial waste comes from industrial pro- 
cessing. The Recycled Grey collection represents a theme already pur- 
sued by Pedrali, who has always reinserted in the productive process 
the byproduct of internal waste of its factories: plastic material scraps, 
containers, and plastic films. The 50% of post-consumer waste repre- 
sents the limit that allows this material to guarantee the product’s ex- 
acting standards regarding durability for the contract sector, their pri- 
mary activity sector, known for its heavy and prolonged use. 

Upcycling describes the process of increasing the value of waste 
materials through the recycling process, creating a product with a 
longer lifespan than the original. Lendager UP is a Danish company 
working in upcycle product development. An agreement between 
Lendager Group designers, world-renowned flooring manufacturer 
Dinesen, and Danish kitchen manufacturer Reform results in UP 
kitchen cabinets which are created from solid wood, post-production 
cutouts that would otherwise have ended up in the landfill as waste. 
UP kitchen cabinets are examples of value gained by upcycling. 


Fig. 3.5 - Sacco’s Green Limited Edition of the iconic chair by Zanotta (2019). 
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3.6.4 Utilize pure materials to improve the quality of post- 
life use 


Utilizing pure materials can impact the furniture sector by extend- 
ing the EoL and improving durability and utility of products. These 
examples identify calculated and conscious decisions to create sustain- 
able furniture options. 

Caimi is a leading Italian company in the production of furnishing 
accessories. Caimi’s Snowsound technology acoustic panels are com- 
posed of 100% recyclable polyester with variable density, which 
achieve selective absorption at different frequencies to optimize their 
acoustic performance. Plastic and metal components are easily detach- 
able and recyclable. 

Zanotta’s remake of the 1969 Sacco has “gone green” and is now 
made of ECONYL®, a regenerated nylon thread made entirely from 
fishing nets collected from the seabed, scraps of fabric, and industrial 
plastic. The new material maintains the same qualities and perfor- 
mance as nylon made from petroleum but can be regenerated, recre- 
ated, and remodeled indefinitely without using other natural resources. 
In addition, the padding of the original project, made of high strength 
expanded polystyrene (EPS) balls, has been replaced with BEWI’s 
BioFoam® microspheres, a biodegradable and compostable bioplastic 
(PLA) obtained from sugar cane that has the same characteristics of 
EPS in terms of structure, properties, and technical performance. 


3.7 Discussion 


The points of action highlighted in the case studies point to con- 
cepts and values that matter in the furniture sector. They point to op- 
portunities that the furniture sector can lead with. Across the furniture 
sector, there appears to be scattered adoption of CE models despite 
broad knowledge and support for the goals and values of CE models. 
Why the gap? 

Designers, educators, business directors, and the press are all in po- 
sition to influence consumer perceptions and consumer patterns that 
can encourage industry to transition away from unnecessary resource 
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depletion, but to do so, attitudes and paradigms must change, and de- 
sign must demonstrate that it can result in more than superficial 
changes to products (Andrews, 2015). Change must be systemic on 
several levels and embrace core aspects in the manufacture of furniture 
to be consequential. 

It was valuable to discover a patchwork of CE criteria adopted by 
European furniture companies, however, quantitative measure is dif- 
ficult and complex to obtain. Therefore, it would be desirable to have 
an agreed upon common set of criteria for the furniture sector, com- 
plementary to the Circular Indicators Project developed by the Ellen 
MacArthur Foundation, and related to the ‘Green Furniture Mark’ 
(GFM). Doing so would provide consumers and procurers a simple 
means of assessing product circularity. Potentially, a GFM could be 
deployed alongside other existing EU instruments, such as the EU 
Ecolabel and GPP criteria. 

There are many potential benefits to transition and adopt CE mod- 
els in the furniture sector. Benefits of a higher level of CE in the EU 
Furniture Sector (EEB, 2017) could result in: 160,000 extra jobs, 3.3- 
5.7 million tons of additional reused/recycled materials (Improving 
carbon footprint), and €4.9 billion increase in EU’s Gross Value 
Added (GVA) statistics. 

Sustainability and circularity drivers will continue to impact man- 
ufacturing processes as well as new business models and new ways of 
production. This is essential to ensure the European industry’s mid and 
long-term competitiveness with implications for workers and their 
safety. A circular economy approach in an environmental context 
could drive innovative strategies to prevent and minimize resource 
consumption, build into the continual product maintenance of materi- 
als in time-based cycles, and recycle potential waste into new uses. In 
addition to the environmental advantages, the adoption of circular 
practices results in financial savings. 

By developing the knowledge base for greener and smarter manu- 
facturing processes, the authors believe it is possible to achieve a sus- 
tainable and more democratic European design culture. Long term sus- 
tainable and green designs are embracing smart and innovative CE 
values involving social, economic, ecological, systems and require a 
process-based and multi-scale systemic approach to planning and 
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realizing sustainability guided by a target vision (Bagheri and Hjorth, 
2007). It is up to us all, designers, industry, institutions, and consum- 
ers, to consider sustainable furniture design strategies and consider cir- 
cular economy models and scenarios in manufacturing when making 
decisions and placing value in furniture design. 


3.8 Conclusions 


Our research on CE solutions and strategies has focused on the 
knowledge gap in the furniture sector. We believe that guidelines and 
assessment criteria will encourage the adoption of CE models for the 
manufacturing of furniture in Europe and can help the sector achieve 
social, economic, and environmental sustainability, and contribute to- 
wards reaching the environmental goal of carbon-neutrality outlined 
in the EU Green Deal. By developing the knowledge base for greener 
manufacturing processes, it is possible to plan for a sustainable and 
more socially democratic European furniture design culture. 

The furniture industry needs to be encouraged to adopt a systemic 
transition from a linear economy to a circular economy because it is a 
more responsible model for achieving the sustainability goals aspired 
by many European institutional entities. To transition across the in- 
dustry, designers and companies must adapt to new economic rules of 
engagement. Additionally, the public sector actors (influencers, me- 
dia, teachers, customers) need to be further educated about the value 
of the CE model. With an educated public who expresses interest in 
C2C products, designers and companies can confidently push the ex- 
isting boundaries of what is defined as sustainable furniture. But this 
should not stall any progress each company identifies for itself to be 
part of a circular economy. 

In short, designers and manufacturers need to shift their thinking, 
by adding [re] into all aspects of making. The takeaway is that there 
appears to be a gap between the positive attitude towards CE systems 
and implementation strategies, which suggests potential growth for 
both institutions and managers involved in sustainable development 
processes. In this context, there is a clear need to study the impact of 
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legislative and voluntary actions in the next decade, forecasting how a 
more sustainable CE will affect and transform the EU furniture sector. 
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4. Design by data in adaptive morphologies 


Attilio Nebuloni, Department of Design, Politecnico di Milano 


Abstract 


Design practices have looked at natural systems as the source of 
inspiration for centuries to emulate their formal aspects and their life 
and evolutionary complexity. Since the advent of the digital revolution 
and the technological innovation related to computation, this issue has 
become increasingly part of the design plan. Today, thanks to the 
code-based generative tools and processes, design research is increas- 
ingly exploring the principles of biology and particularly the aspect of 
adaptability, which means “adaptive,” the ability of an organism to 
transform its original structure into a different one relating to an envi- 
ronmental condition, and thereby designing a symbiotic relationship 
between body and surrounding context. As a result, this kinetic and 
adaptive matrix provides strategies for designing and constructing 
emerging morphologies and design elements to experience creative 
possibilities linked to the relationship with the environment. 

The chapter proposes a conceptual framework for the topic’s ex- 
ploration in the context of computational design, that is a multidisci- 
plinary area of study which can be defined as the application of com- 
putational strategies to the design process and whose relevant aspect 
concerns the creative, logical nature, and not the mere instrumental 
component of the “calculation”. The essay also aims to encourage re- 
thinking the project’s territories, viewed as growingly hybrid context, 
where boundaries of the design’s fields (mainly design, architecture, 
art, engineering, etc.) can virtuously intertwine with digital subjects of 
computation and therefore acquiring and increasing design value. 
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4.1 Introduction 


Over the last decade, design has increasingly taken advantage of 
the growing availability of digital technologies of a computational na- 
ture, revisiting and reformulating previously little addressed problems 
due to their complexity, both in terms of the number of components 
and the relationships between them. It has led to concepts, techniques, 
and tools such as algorithmic design, parametric design, generative 
design, digital manufacturing, and, more recently, design by data. The 
latter, specifically, comes from representation and generative method- 
ologies and deals with parametric definition and algorithmic pro- 
cessing. Its focus is the use of data from real-time sources, but not only 
to develop scenarios capable of integrating the variability of the data 
themselves, both in the dynamics of the simulation as in the adaptive 
capacity of the physical systems of a built environment. Computer sci- 
ence and new technologies have provided the main paradigms of the 
subject, while the design has outlined new models and research meth- 
odologies. 

The reference context for research is the computational design, un- 
derstood as the design process that, taking advantage of the potential 
of computation, integrates digital and emerging technologies in the de- 
velopment of a product structured on the interaction between form and 
information (Reas, 2010). In this framework, the programming and 
design domains integrate to identify a form of creativity where infor- 
mation becomes procedures and rules of interaction. Furthermore, 
where data drives a design transition from descriptive to prescriptive 
approaches establishes an unprecedented feedback loop in design re- 
search and production. 


4.2 From data to computational design 


The architecture has always made use of data in the design process. 
Specifically, data made of different nature and from various sources, 
which determined the premises, guided the construction, and reviewed 
and implemented phases. The quantitative and functional data are 
those that usually define the beliefs. In contrast, the qualitative data of 
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the relationship with the environment (whether intended as context, 
people, etc.) structure the articulation, characterizing a specific mor- 
phological and spatial interpretation. In the context of this reflection, 
“set of data” means a collection of “raw” values, not determined to a 
specific form and scale, which through a process of helpful refinement 
for their interpretation (a method of translation within a particular do- 
main of reference), become materials through which to build infor- 
mation or structure knowledge. In the case of architecture, this struc- 
turing process leads to composition. 

The translation of data into representation has thus a double role: 1) 
to build a sort of general framework of the project, which defines the 
framework of rules and the general principles of interaction; ii) to 
make the project emerge from an articulated whole, but not yet 
formed, which already possesses within itself all the elements that will 
characterize its generation. 

Converting data from external domains into a project structure is a 
problem of rewriting in a different language and equivalence between 
the parts since the system of signs strictly connects its interpretation 
within other signs and semantic contexts (Fig. 4.1). 


Fig. 4.1 - Simulation of the responsive behavior implemented by facade petals in the 
Aegis architects’ project Al Bahar Tower in Dubai (author ’s revision of façade de- 
tail). The algorithm determines the dynamic morphology of a star origami by calcu- 
lating the distance of each center from the sun ’s line of incidence. The transversality 
of the data is the crucial factor in the design. The data obtained are continuously 
remapped between 0/1 domain, corresponding to the opening and closing positions 
of the petals, then used as input in the control of the distance between the points of 
the geometry. 


As Speed & Oberlander (2016) suggest, it is possible to distinguish 
three main approaches of the relationship between design and data: 
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i) design from data, ii) design with data and iii) design by data. These 
three classes, very different from each other, are defined by observing 
the flow of data. From a condition of substantial static and independ- 
ence between the two domains (design vs. data), progressively changes 
to another more dynamic and interactive state that interprets the vari- 
ability of the parameters as a valuable element to explore emerging 
and non-linear design suggestions. 

Specifically, the approach that characterizes the design from data 
relates to those design systems entirely guided by the designer, who 
then identifies, selects, and draws inspiration from inputs obtained by 
measuring known elements of the context. And regardless of the meth- 
ods (observation, interviews, tests, etc.) and tools used in such meas- 
urement (analog vs. digital). The data is consequently a premise of the 
project. 

On the other hand, in the design with data approach, the process is 
still characterized by designers’ determination to guide the implemen- 
tation of the phases. Still, also it is flanked by data flow from interme- 
diate levels of verification and control of processes/products. This pro- 
cess is not only to collect cold elements of experimental synthesis, 
useful in the design review but “a condition in which designers should 
anticipate the disruptive potential that is produced from streams of live 
data from networked artifacts” (Speed, Oberlander, 2016). In this case, 
data is played out between the premises and the implementation 
phases of the project. 

With the approach of design by data, finally, are collected most of 
the experiences of the other two classes to project them into a new 
design dimension and in which systems with large margins of auton- 
omy and not totally defined in their morphologies are governed by 
other systems, which interact to generate information. Thus emerge: 
“new products and services can be synthesized via the data-intensive 
analysis of existing combinations of humans, computers, things, and 
contexts” (ibid.). The condition of the data is, in this case, dynamic, 
and the project is structured on open variables. 

The design from data represents the most classic and traditional ap- 
proach to the project, in which the linearity of the process is the de- 
signer’s responsibility. On the other hand, design with data is an inter- 
mediate phase where this linearity is questioned, and the project opens 
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to interaction with the context. Finally, it is possible to speak of design 
by data when the importance of the data defining the project domain 
(constraints, requirements, etc.) is significantly marked. Results are 
therefore derived by developing an algorithm and leveraging new dig- 
ital technologies and computational science tools. From data, we ar- 
rive at the project through the computation (here understood as the 
algorithm of a parametric function) that defines the behavior and con- 
figuration of the space when a set of parameters (geometric, logical, 
environmental, etc.) changes (Fig. 4.2). 


ae 
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Fig. 4.2 - The Aegis Hyposurface (© dECOi) marked a transition from determined 
to interactive morphologies. It was a faceted surface able to deform its morphology 
as a real-time response to environmental inputs: sound, video source, user input, or 
by pre-configured effects. The surface is made by aluminum facets glued to rubber 
articulations, able to ensure a fluid visual continuum. A set of pistons drove the fac- 
ets to generate a dynamic surface as a response to electronic calculation. 


Emerging from the same physical space and employing real-time 
values, these morphologies may continuously reconfigure or adapt to 
follow environmental and user needs. Variability is what marks their 
character. This aspect does not relate to the final form of the physical 
object but to its being an open and indeterminate structure (expressed 
by the ability to vary along a curve of possibilities), which with this 
artifact shares the origin of its organization!. 


! Bernard Cache (1995) explains a generic design entity with the concept of objective, never 
completely defined, and more important than its specific elements. The antithesis to the 
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4.3 From computational design to adaptive morphologies 


One of the critical strategies of the computational approach is to 
“think algorithmically” about the project or to code in the structure of 
an algorithm, the system of relations that links data to output by pro- 
cessing (Werner, 2015). 

With the adoption of the computational approach, the designer does 
not necessarily leave traditional skills and experiences. Nevertheless, 
it is necessary to redefine them in terms of objectives, languages, and 
operating structures; from the mere representation of a single static 
configuration, the focus shifts to constructing and managing the sys- 
tem of relations that develops the project. Moreover, where the output 
represents only one of the n-possible interpretations. Thus, the focus 
of the project task becomes the definition of a design environment sep- 
arated from the physical representation of its geometry, i.e., a space 
conceived as a system of relationships, in which the project is the place 
where a series of parameters generate design iterations. 

The gain offered by the definition of the problem in algorithmic 
terms allows to generalize the problem itself and, therefore, to transfer 
it to similar contexts, in the way as everyday objects find a diversity 
of interpretations to the context within which they are inserted, rede- 
fining from time to time their skills. 

Here is the main difference between classical to algorithmic mod- 
els: design with algorithms means defining “an architecture of the ar- 
chitecture,” and raising the design problem to its generalization finds 
the abstraction necessary to go beyond a single specificity. 


finished object of industrial production follows a specified time, coded use, uniqueness, and 
invariance of the model to be reproduced. The peculiarity of the objectile is its potential to 
study an open set of morphologies, where the variety principle (distinctive element to be en- 
hanced and not an imperfection to be eliminated) is more important than homogeneity by 
options. In order, the concept of objectile Mario Carpo (2011) states that designer’s attention 
shifts from the autopoiesis vision of form to process and numerical matrix at the origin of an 
open and imperfect object, expression of a function rather than a finite image of geometry and 
space, able to restores roles and competencies: «new digital platforms for open-ended, inter- 
active collaboration may beget endless design variations, revisions or versions, loss of design 
control and authorial recognition, and even, in the most extreme cases, collective or anony- 
mous results (...) They may design objects, and then be digital interactors. Or they may design 
objectiles, and then be digital authors» (Carpo, 2011). 
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Alongside this, the spread of digital technologies and algorithmic 
modeling tools extends the ability to simulate and predict the behav- 
iors of design solutions developed in the digital model. The oppor- 
tunity is mainly due to the enabling possibilities of technologies to in- 
teract with the context, leaving potential internal configurations to 
change in design from an initial condition (Fox, 2013). Data can en- 
code information and become a source to simulate behaviors and 
shows adaptive outcomes in time: from representation - 2D, or para- 
metric modeling - 3D, design becomes a generative simulation process 
- 4D (Bier and Knight, 2014). Derived from biology, with “adaptiv- 
ity,” we mean the ability of an organism to modify in whole or in part 
the elements of its original structure to a different one when specific 
environmental conditions change. 

In architecture, adaptive systems are defined as complex systems 
that integrate both hardware and software components. The program- 
ming (coding of an algorithm) defines the rules of transformation re- 
lated to environmental inputs. Hence the dynamic behavior is the ad- 
aptation of its morphology to the environment to which it refers. As it 
happens in natural systems, where changes in environmental condi- 
tions shape a slow and continuous process of adaptation of their struc- 
tures, so in the design of adaptive architectures, the adoption of these 
principles leads to the emergence of artifacts and spatial organizations 
capable of setting their morphology on the dynamics of action and re- 
action to a series of external inputs (Fig. 4.3). 

At the level of the design process, as well as the skills required to 
meet this challenge and expected results, there is a “technological hori- 
zon that needs a new culture, capable of hybridizing immaterial and 
material aspects” (Campioli, 2020). These aspects overcome the com- 
positional logic of the whole object and the “solidity” of the building 
system. Because of the need to respond to specific local conditions, as 
opposed to generic overall configurations of the composition ordered 
by geometry, these artifacts are often described as non-standard archi- 
tecture, and: «employ the building envelope and its articulation and 
multiplication as a spatial device and environmental modulator» (Hen- 
sel and Sørensen, 2014). 
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Fig. 4.3 - The Resonant Chamber project (© rvtr) was iterative research and devel- 
opment in computational testing and full-scale prototype installation for the Univer- 
sity of Michigan. The project was an exploration of kinetic tessellated architectural 
systems and variable acoustic surfaces that use specific geometries. The project 
aimed to develop a sound sphere able to adjust its tessellated origami patterns to 
achieve the optimal conditions for the sound. 


The context outlined by the two authors is hybrid, and the relation- 
ship between analog and digital takes on peculiar features. Moreover, 
where the different disciplinary fields (mainly design, architecture, art, 
engineering) intertwine with each other and with the topic of digital 
computation to acquire and increase design value. Boundaries of the 
various domains and the rigidity of the scale that drive additive com- 
positional logic between the different layers of the construction are 
blurred (Nebuloni, 2020a). The surface is first discretized into basic 
modules and then populated by new families of technological artifacts 
from an organic whole: the adaptive component system. 
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According to the level at which the relationship between design and 
data takes shape, these artifacts’ adaptive systems are recognized as 
central and local. The former defines the control of the set of compo- 
nents on the main dimension of the structure (Fig. 4.4). The latter 
keeps it within the single element that, in this case, will be equipped 
with sensors, actuators, and microprocessors for data encoding and 
motion control (Fig. 4.5). 


Fig. 4.4 - Soma Studio. Thematic Pavilion for Yeosu Expo 2012 (©Soma). The de- 
sign’s concept for the 140 meters long and 3 to 13 meters height dynamic facade 
originated from biology and is an example of a central level system. The 108 kinetic 
lamellas controlled internal light conditions and created organic external animated 
patterns. Servomotors reduced the distance between the two bearings to originate 
the movement, resulting in a side rotation of the lamellas. 
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Fig. 4.5 - Jenny Sabin Studio. Lumen, New York (© Jenny Sabin Studio). Lumen 
consisted of 250 hanging tubular structures designed to capture and react to 
changes in sunlight over the day. The concept originated from biology, and its com- 
position refers to a local-level systems typology. The structure’s reactive surface 
was linked to human interaction and environment. 


Between the two systems, there are essential differences in the ty- 
pological structure and data management: in the central systems, the 
scale of the components is greater, and the control over the whole pro- 
cess, as well as the harmonization between the elements themselves, 
develops more complex kinetics. In the local systems, the components 
are smaller and with a form of intelligence that allows them to produce 
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individually simple movements and achieve complex and unexpected 
overall results. 

The analogy is to the difference that occurs in nature between the 
plant systems - with a disseminated intelligence and largely autono- 
mous -, and animal systems - where the control is at the central level. 
In the first system emerges a logical relationship between homogene- 
ous parts, while in the second, a structure of superordinate elements 
and succession between them. 

Therefore, the characteristic of adaptive architectures is an aesthet- 
ics of movement resulting from the ability of action of a plurality of 
dynamic artifacts like each other. The form - or rather the configura- 
tion, potential and induced by data - is a function of the parameters’ 
degree, and a program (algorithm + coding) governs the behavior of 
the system: the algorithm founds the logic, while the code implements 
it in a formal language (Nebuloni, 2020b). 

Most modeling software integrates interfaces or plug-ins (apps that 
add advanced functionality to the essential software) to explore these 
morphologies in the project. With proper programming languages, 
these interfaces allow the customization of the modeling environment, 
the definition of algorithms, and control data. In these environments, 
the algorithm often uses visual programming paradigm, mainly 
through block and flow diagrams. 

These skills also enable interoperability between different digital 

design environments and platforms, which is essential for integrating 
external inputs into the process. 
The result is the communication and interaction between different do- 
mains (e.g., Grasshopper and Firefly plug-ins) that connecting in a bi- 
directional way an algorithmic design environment to a microcontrol- 
ler (e.g., Arduino). Data from sensors are, therefore, processes to sim- 
ulate the physical behavior of a component. Virtual environment and 
a real physical one are both engaged (Fig. 4.6). 
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Fig. 4.6 - Digital-physical interaction via a prototype. Simulation of the responsive 
behavior of a design component (Responsive morphologies Workshop, Polimi). 


4.4 Conclusions 


Over the past few decades, digital technology has already contrib- 
uted significantly to the redefinition of design practice.” Today, the 
greater diffusion of tools, techniques, and languages capable of inte- 
grating programming into modeling and the possibility for the de- 
signer to build custom working tools foreshadow even more important 
changes. Among them, in addition to the ability to analyze a plurality 
of data and information proper to computation, the role of enabling 
technologies in connecting environments and simultaneously 


? There are three design research stages in the recent history of the digital revolution in archi- 
tecture (Wiscombe, 2015): the first decade of openness (the nineties), characterized by the 
free and seemingly unconstrained forms brought by new tools and techniques, from which a 
figurative apparatus made of hybrid and not rigidly defined morphologies, far from the refer- 
ences of history and no longer articulated on the composition of the parts, but based on the 
processes of generation linked to the matter, has derived; the second (first decade of the two- 
thousands), markedly computational, of experimentation on the forms of programming and 
modeling algorithms, in a game of scale between the variability of a single element and the 
projection of the same on the morphology of the whole; the current (from 2010), which rein- 
force the loyalty to the relational aspects of the project focuses on adaptive objects and the 
dynamic logic of algorithms. 


79 


Copyright © 2021 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788835125716 


managing many design options. New references, resulting from the 
changing relationships between environment, construction, and de- 
sign, are followed by the definition of new interpretative frameworks 
focused on the behavior of systems and the ability to connect contents 
and exchange information. Combining and harmonizing the skills nec- 
essary for effective and creative use of digital technologies is increas- 
ingly needed to explore the potential of the relationship between de- 
sign and data, implying that designers “reset” the classic tools of the 
project to build new languages based on variation and complexity. 

There are still many risks and obstacles that characterize this hybrid 
and creative phase of the digital project to make definitive statements 
about its viability. Above all, the relation to cultural references and 
examples that, although promising, identify architecture as a mere 
space of representation of concepts far from its disciplinary domain. 
Therefore, it will be essential to broaden the discussion to include: 
“beyond ontology and epistemology, the key issues of function, typol- 
ogy, and technology” (Wiscombe, 2015). Not least, to combine per- 
formance aspects with an aesthetics of movement leading to a defini- 
tion of form that is increasingly dependent on dynamic environmental 
variables. 
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5. Developing Interactive Architecture Prototypes 
by Means of Design-to-Robotic-Operation 


Henriette Bier, TU Delft 


Abstract 


Concepts for Interactive Architecture have been developed since 
the 1970s and more recently with advancements in Cyber-physical 
Systems and the Internet of Things these concepts have increasingly 
been developed towards practical applications. This chapter presents 
Interactive Architecture applications developed at Technical Univer- 
sity Delft using Design-to-Robotic-Operation methods. They are proof 
of concept for the potential of interactive systems to seemingly disrupt 
the linearity of time and improve spatial experience. 


5.1 Introduction 


In the 70s (inter al. Eastman, 1972; Negroponte, 1975) speculation 
on opportunities introduced by the digital technologies initiated a dis- 
course on intelligent environments in architecture. Since then, various 
applications have been developed for Ambient Intelligence (Aml) 
(Zelkha et al., 1998), Interactive Architecture (inter al. Fox & Kemp, 
2009), Adaptive Environments (Bier, 2018), etc. 

In the same line of experimentation, the Design-to-Robotic-Produc- 
tion- Assembly and -Operation (D2RPA&O) framework developed in 
the Robotic Building (RB) lab at Technical University Delft (TU 
Delft), explores the potential of robotic and cloud-computing technol- 
ogies and their integration into buildings and building processes (Bier 
et al., 2018). D2RPA&O makes use of Cyber-physical Systems (CpS) 
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and the Internet of Things (IoT) in order to monitor physical processes 
by creating virtual representations of the physical world that support 
decentralized decisions making (Rajkumar et al., 2010). Thus, 
D2RPA&O relies on (a) interoperability, which is the ability of robotic 
systems, humans, and buildings to connect and communicate via the 
IoT, (b) virtual-physical coupling by linking sensor-actuator data 
(from monitoring physical processes) with virtual models and simula- 
tions, (c) decentralization, which exploits the ability of cyber-physical 
components to operate autonomously, and (d) real-time operation im- 
plying that data is exchanged in real-time. 

Several case studies focusing on D2RO as part of the larger 
D2RPA&O framework will be presented. Their aim is to extend hu- 
man capabilities and experiences in the built environment by means of 
sensor-actuators. Even if different from D2RP&A, D2RO is seen in 
terms of the processes involved as very much similar because they all 
involve some level of automation and Human-Robot Interaction 
(HRI). Similarities and differences will be identified in the sections 
‘Case studies’ and ‘Implementation’ with the section ‘Conclusions’ 
framing achievements so far and identifying future steps. 


5.2 Case studies 


Several case studies are presented in which virtual and physical 
worlds communicate with each other: 


5.2.1 Interactive Fagade (2019) 


The first case study has been implemented as workshop at TU Delft 
with the goal to develop a façade that responds to human and environ- 
mental needs. While people’s movement and posture was recognized 
using Kinect, an array of sensors provided ambient lighting data. This 
data combined with weather data throughout the day provided the ba- 
sis for predictive analytics. Multiple Arduinos were distributed along 
the façade and work as an intelligent swarm to implement reconfigu- 
ration of the fagade according to sun conditions and users’ movement 
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Fig. 5.2 - Linking accelerometer with virtual representation. 
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in space. When linked to Artificial Intelligence (AJ) in order to able to 
learn from the data accumulated in time, the system improves and pro- 
actively proposes reconfiguration of the facade by increasing or de- 
creasing shading, transparency, etc. It not only connects the virtual and 
physical environment in a functional but also a playful way, which has 
been the focus of the next case study (Fig. 5.2). 


5.2.2 Hiperorganicos (2018) 


Organised by Nucleo de Arte e Novos Organismos (NANO), Fed- 
eral University of Rio de Janeiro as part of the Hiperorganicos sym- 
posium (2018)! the workshop offered by TU Delft focused on the de- 
velopment of sensor-actuator prototypes linking virtual and physical 
worlds. Participants developed various prototypes from which one is 
showcasing a ball on strings linking its physical movement in space 
to its virtual representation thus creating a virtual-physical continuum 
(Fig. 5.3). 


Fig. 5.3 - Accelerometer embedded in ball on two strings is used link the physical 
movement with its virtual representation. 


! https://nano.eba.ufrj.br/hiper8/. 
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Using sensor-actuators and processing? the physical ball is linked 
with the virtual. Each movement of the ball initiates transformations 
in the virtual world. The virtual and physical worlds mirror each other 
in a playful way. While there is only one physical ball moving in a 
controlled trajectory of up and down, the virtual world represent mul- 
tiple balls showcasing the artistic interpretation of the trajectory. 

By introducing Design-to-Robotic-Operation (D2RO) framework 
developed at TU Delft workshop participants design explored the use 
of Wireless Sensor and Actuator Networks (WSAN). Projects aimed 
to interface the virtual and physical worlds via intuitive mechanisms, 
where physical actions mirror actions or reactions in the virtual envi- 
ronment. The project presented here used physical motion to define 
the position of digital objects within a virtual space (Figs. 5.2 and 5.3). 
In this project, a wearable device integrated into a ball on two strings 
was used to link accelerometer data wirelessly to a virtual environ- 
ment. This data informed the virtual environment to mirror the physi- 
cal movement within the physical space. By connecting the physical 
with the virtual, the architectural space transforms into a hybrid envi- 
ronment, wherein spatial experience is not anymore sequential. 


A passage The È A 
through the connection al 
landscape between twe eM 

highs landscape 


cantiliver 
bove the 
structure 


he lowest 
surface of the 


Fig. 5.4 - Landscape-like design based on activities 24/7. 


? https://processing.org. 
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5.2.3 Interactive Urban Furniture (2017) 


The third case study explores the potential of interactive systems for 
urban furniture? with movement sensors and lighting actuators inte- 
grated into a landscape-like design located at the river side in Rotter- 
dam. The study of activities and potential new activities on respective 
location played an important role in the design (Fig. 5.4). In addition, 
factors such as climate, water tides, surrounding urban context, flora 
and fauna, etc. were studied. Once the parameters to inform the design 
were identified, the form-finding process was initiated using activity 
patterns as driving force. Voxels were employed to define the scale 
and distribution of activities in each part of the urban furniture and 
then their smoothening was implemented in order to achieve a contin- 
uous easily accessible landscape. 


Fig. 5.5 - Sensor-actuators integrated in the urban furniture. 


The integration of sensor-actuators was implemented in relation- 
ship to the identified and newly defined activity patterns. Lights were 
integrated to not only assist users with their activities but also instigate 
playful interaction in form of ‘catch me if you can’ or similar. The 
lights would turn on and off, change intensity or color depending on 
the activities that are taking place and the time of the day. They would 
precede or follow runners or surround resting users. 


3 http://uf.roboticbuilding.eu/index.php/Msc1G3:Group. 
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When enhanced with AI, the system is envisioned to be able to learn 
in time based on users’ behavior. The system may start with a basic 
set-up that improves in time based on collected data (Cheng et al., 
2017). The system may become in time more attune with local condi- 
tions and users’ needs. Users may give as well direct input via an app 
so that the system is able to recognize specific users and adapt to their 
individual needs, perhaps, even initiate alternative interactions that us- 
ers could appreciate. 

In this context, the space becomes an active participant in the spa- 
tial experience of the user, disrupting the linear experience of space 
and time in architecture. Spatial experience changes in time while us- 
ers move through space. By integrating robotic, AI, and IoT applica- 
tions into architecture, the design, production and operation of physi- 
cally built environments establish an unprecedented feedback loop, 
which has been further explored in a project with focus on Ambient 
Intelligence (AmI). 


5.2.4 Interactive Podium (2016) 


An extended AmI enabled by a Cyber-Physical System (CPS) built 
on a Wireless Sensor and Actuator Network (WSAN) has been devel- 
oped (inter al. Liu Cheng et al., 2017; Bier, 2018) by integrating 
amongst others, Human Activity Recognition (HAR), in order to reg- 
ulate interactively inter al. illumination (Fig. 5.6). Integrated in a po- 
dium with LED-lights the system responds to three states: (1) Initia- 
tion, (2) Lecture, and (3) Break. In the first state, the lights start to 
pulsate indicating that the podium goes through a starts up procedure. 
In the second state, the lights react to the speaker, by change color and 
intensity. In the third state, the lights respond to the audience and their 
movement during the break. The system is equipped with Machine 
Learning (ML) algorithms in order to identify which combinations of 
light color and intensity contribute to the comfort of the speaker. 
Hence, the system aims to improve the state of the speaker by regulat- 
ing the light via ML mechanisms using HAR (Liu Cheng et al., 2017). 
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Fig. 5.6 - Interactive stage at the Game Set Match symposium 2016. 


5.2.5 Drones Swarm 


Moving beyond integration of interactive devices into the built en- 
vironment, in a project developed for a fictive world expo, students 
designed swarms of drones that guide visitors through the exhibit site 
and create temporary dome-shaped pavilions (Fig. 5.7). The swarm of 
drones was conceptually designed to operate semi-autonomously. The 
drones rely on Swarm Intelligence (SI) to act as a coordinated swarm. 
They follow simple rules of separation, alignment and cohesion and 
are communicating with each other while being as well aware of the 
environment hence avoiding collisions and aggregating in specific 
configurations. 


Fig. 5.7 - Swarms of drones reconfiguring according to human needs rely on a bio- 
cyber-physical feedback loop. 
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In this context, developing ML models in order to introduce capa- 
bilities of learning in time, is a challenge. If SI works with data col- 
lected within a short period of time, ML employs data collected from 
users and environment over a longer period of time with the aim to 
learn to respond to users’ needs by establishing a bio-cyber-physical 
feedback loop (Fig. 5.7). Such feedback is required mainly because 
Cyber-physical Systems (CPS) are designed to be autonomous but pre- 
sent uncertainties with respect to complex decision-making, trouble- 
shooting, etc. (Pillan et al., 2020). The Bio-CPS model is integrating 
computational and physical elements with biological systems (Fass 
and Gechter, 2015). Such an approach ensures that the built environ- 
ment actively responds to human needs. The responsive environment 
is sustainable because it learns from users and its functionalities are 
attune with what is needed (Liu Cheng et al., 2016). As a result space 
is more comfortable, use is more efficient, and energy loses are re- 
duced. 

Such ML algorithms employ data collected from users to learn how 
to respond to users’ needs by establishing a bio-cyber-physical feed- 
back. The design of such feedbacks requires D2RPA&O methods 
(Bier et al., 2018) that learn from users and the environment. This 
learning process takes place in both the D2RP&A as well as in the 
D2RO processes. While D2RO links the design to the operation of 
buildings, D2RP&A focuses on linking the design to the production 
and assembly process of buildings (Fig. 5.8). Together, they establish 
a comprehensive framework for AI supported building of buildings 
that are imbued with AI. Both, AI embedded in building processes 
(based on D2RP&A) and AI embedded in buildings (based on D2RO) 
involve on some level Human Robot Interaction (HRI). If in D?RP&A 
processes, humans work safely together with production robots, in 
D2RO humans interact safely, healthily, and pleasurably with the ro- 
botic built environment. In both cases the new meaning of building 
production and operation is not created by the one or the other, but by 
the interaction between the two. 
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Fig. 5.8 - Human assisted robotic assembly by stacking. 


So far AI has been integrated mainly in D2RO (inter al. Liu Cheng, 
2017; Bier et al., 2018) and more recently the integration into the 
D2RA process has started (Fig. 5.9) with the robotic arm using ML, 
and computer vision techniques, such as OpenCV, to find location of 
nodes, detect the related linear elements, pick them with a gripper and 
transfer them to the intended location (in the next proximity of the 
node) one by one, while considering obstacle avoidance (human 
safety). In this scenario, the human will navigate the arm with his/her 
hand, and in order to move it to its final location (the node). For that 
the following steps are considered: (a) Localization by creating a map 
of the environment (including nodes, linear elements, and human lo- 
cation); (b) Robot’s location by object detections (using OpenCV to 
detect the correct node, and do the corresponding action); (c) Control- 
ling and navigating the gripping toward the objects in order to pick up 
the objects; (d) Human action involving controlling the gripper manu- 
ally to insert the linear element in the node. 

The overall goal is to let the tasks that are more easily implemented 
by humans in human control while the others are taken over by robots. 
The question of what tasks are automated or semi-automated in D2RA 
has been first formulated in a human-assisted robotic assembly exer- 
cise that involved staking. 
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Fig. 5.9 - Computer vision approach (right) developed for human-robot collabora- 
tive assembly of linear elements and node (left). 


5.3 Human-assisted robotic stacking 


The project focused on human assisted robotic assembly involving 
stacking linear elements with varied sizes in multiple directions (Fig. 
5.8). Several processes of parametrization, structural analysis, and ro- 
botic assembly were algorithmically integrated into the D2RP&A 
method. This method was informed by the systematic control of den- 
sity, dimensionality, and directionality of the elements. As all other 
processes, it was tested by building a one-to-one prototype, involving 
development and implementation of computational design workflow 
coupled with robotic kinematic simulation that is enabling the materi- 
alization of a multidirectional and multidimensional assembly system. 

The virtual and the physical processes were linked in all phases 
from design to production and assembly, while humans and robots 
worked side-by-side requiring consideration for HRC, which is now 
being developed. 


5.4 Implementation 
These case studies were developed as combined research and edu- 


cation projects taking advantage of synergy effects between research- 
ers, PhD and MSc students, who were gradually familiarized with the 
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use of computation and robotics embedded into buildings and building 
processes. 

D2RPA&O builds up on interaction between human and non-hu- 
man agents not only at design and production level but also at building 
operation level, wherein users and environmental conditions contrib- 
ute to the emergence of multiple architectural configurations (inter al. 
Bier, 2018). These utilize sensor-actuator mechanisms that enable 
building components and buildings to interact with their users and sur- 
roundings in real-time. Their conceptualization and materialization 
process requires D2RPA&O chains that link design to production, as- 
sembly and operation of buildings. In this context, design becomes 
process-oriented and use of space is time-based, which implies that 
architects design increasingly processes from which sensorially or/ 
and physically reconfigurable buildings emerge. 

As presented in the case studies, reconfiguration serves a variety of 
purposes and requires advanced virtual modeling and simulation that 
interface the production and real-time operation of physically built 
space through D2RPA&O. 

As earlier indicated, the virtual and the physical processes are 
linked and need to take into account the inherent multifaceted nature 
of building from the early design to the latest building operation phase. 
Challenges in terms of scale, multi-tool and multi-robot production 
and operation need to be examined in order to achieve chained pro- 
cesses by linking virtual models (such as Rhino 3D model with Grass- 
hopper plug-ins such as Millipede, Ladybug, etc. for simulating struc- 
tural and environmental performance) with robotic devices. The aim 
is to develop/ implement chained D2RP&O processes in which robots 
take specific roles while all (human and non-human) members of the 
setup including respective (virtual and physical) systems receive feed- 
back at all times (inter al. Nazzari and Bier, 2020). 

D2RPA&O has been so far developed with the vision in mind that 
building processes and buildings will be increasingly incorporating ro- 
botic systems supported by AI. The question for the future is thus not 
if but how D2RPA&O will improve building processes and buildings 
while addressing societal challenges such as scarcity, overpopulation, 
climate change, etc. So far D2RPA&O has proven to improve mate- 
rial, energy, production and operation efficiency. Mid- and long-term 
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automation of tasks and processes that can be automated will be im- 
plemented in a similar way as the car industry has automated its tasks 
and processes. Then HRI and ML will be increasingly incorporated in 
processes. The human remains involved in tasks that are requiring sub- 
jective assessment or high-level strategic planning. 

In this context, architecture and building construction fundamen- 
tally change as buildings and building processes become parts not only 
of an urban network but also of a larger Internet of Things (IoT) which 
enables not only production steps but entire value chains to be opti- 
mized, whereby all phases of the life cycle from idea to development, 
production, use and maintenance up to recycling are considered. This 
ensures the development of sustainable approaches that address to- 
day’s societal challenges and improve daily life. 


5.5 Conclusions 


Presented projects identified opportunities to engage in the integra- 
tion of CpS in architecture, while promoting visions of not only func- 
tional but also playful interaction. Such interactions seemingly disrupt 
the linear reading of time when deployed in the mixed-reality built- 
environment. They rely on today’s virtual-physical continuum and are 
indicative of tomorrow’s interactive environments, wherein humans 
interact with CpS that are endowed with increasingly higher levels of 
intelligence. 

While D2RP&A and D2RO are fundamentally different, since the 
one focusses on building production and the other on building opera- 
tion, they are similar in the way the tasks required in the production 
and operation processes are automated. Considering that about 50% of 
all tasks can and will be automated, while 45% will rely on HRI and 
5% will remain in human hands’, the question of impact on architec- 
ture and society at large needs to be addressed. Automation is offering 
solutions that make the design, production, assembly and operation of 
buildings more efficient. At the same time, automation rises questions 


4 https://www.mckinsey.com/featured-insights/digital-disruption/harnessing-automation-for- 
a-future-that-works/de-DE 
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related inter al. to the use of personal data, the social impact from ex- 
pected labor skill shift due to increased automation and the challenge 
to respond to the demand of developing new skills (Pillan et al., 2020). 

Considering the expected labor skill shift, the question is how aca- 
demic education responds to the challenges of automation and the sub- 
sequent requirement that architects develop new skills. Contrary to the 
preconception that robotization is not architecture related in its es- 
sence, the challenge is to understand the same way the modernist ar- 
chitects understood that the industrial revolution 2.0 changed architec- 
ture, that industrial revolution 3.0 and 4.0 with their robotic, AI, IoT 
applications have a major impact on architecture transforming not only 
its design and production but also its operation towards becoming a 
cyber-physically enhanced (production and operation) system (Bier, 
Chan, Cervone and Makaya, 2021). 
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6. From smart materials to Animate Objects. Re- 
framing smartness through behavior exploration 


Marinella Ferrara, MADEC, Department of Design, Politecnico di 
Milano 
Alessandro Squatrito, Alessandro Squatrito Design 


Abstract 


Today, a new class of materials and objects are entering our daily 
lives, giving interest in smart devices research and robotics in our so- 
cieties. As a result, we are experiencing a renewed interest in “material 
agency” and the reconfiguration of our environment through the hy- 
bridization of physical matter and digital performance. 

The purpose of this paper is to highlight the research direction of 
MADEC - the Material Design Culture Research Center in the Design 
Department of the Polytechnic University of Milan - on the phygital 
evolution of product design and the design implications of animate 
objects, which we will define. Since technology has allowed the tran- 
sition from graphic interfaces to shape-shifting ones, designers are ex- 
periencing new challenges. We frame the topic of smart evolution of 
objects through the existing literature, and investigate issues like their 
new features, behaviors, and aesthetic implications. Then we analyze 
a few case studies to stress the fundamental aspects to consider when 
approaching the design of animate objects. 

We believe it is time to strengthen design-led research on behavior 
of animate objects. Their qualities cannot be detached from design as- 
pects inherent to their tangibility during interaction, such as form in 
time and expressiveness. These aspects shape users’ behaviors in 
terms of new functions as well as cultural, social, and ethical dimen- 
sions that are at the basis of collective meanings and values. 
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6.1 The new phygital materiality 


Materials, one of the fundamental elements of design culture (Man- 
zini, 1986; Raimondo, 2006; Ferrara, 2017), have today acquired new 
importance in qualifying the intelligence of artifacts. 

Recent advancements in the materials field - rapid manufacturing, 
electronics miniaturization, computational tools, programmable mat- 
ter, modular micro-robots, and integration - are introducing new de- 
sign variables, enabling researchers not only to apply advanced mate- 
rials but also to conceptualize, program, develop, and then use new 
smart material systems. 

Conventional materials (fabric, plastic films, leather, wood, etc.) 
tightly integrated with interactive technologies (sensors and actuators; 
connection modules; inexpensive, small microprocessors, etc.) have 
given birth to, what we define as, a new phygital materiality (phygital 
comes from physical + digital), which in a previous study we named 
“ICS-Materiality”, where ICS stands for interactive, connected, and 
smart (Ferrara et al., 2018). 

Mainly applied in robotics, smart material systems affect the design 
of new artifacts thanks to their abilities to sense, actuate, and perform 
local computing, as well as their communicating elements (McEvoy 
and Correll, 2015). Moreover, these materials have been shown to 
have a certain intelligence and autonomy, as well as a special formal 
performance, i.e., the ability to change shape and appearance and 
thereby impact applied research, generating “Shape-changing Mate- 
rial Interfaces” (Coelho and Zigelbaum, 2011). This ability generates 
an effect that, if it not the same it is at least comparable whit what in 
context of the performance art, dance and theater has been defined 
“digital performance” (Dixon, 2007, p. 3), i.e. a performance where 
technology «play a key role [...] in content, techniques, aesthetics, or 
delivery forms». 

The phygital materiality with its behaviors, 1.e., its diverse forms in 
temporal and spatial structures, has a fundamental role in “material 
agency”! (Van Oyen, 2018) as an active player in the interactive 


' The concept of agency is vital to understanding the productive nature of material-discursive 
practices, including technoscientific ones. 
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process. Agency is not a property of each user, and neither is it a pre- 
given property of an artifact or material. It doesn’t merely depend on 
material and sensorial qualities either; rather, agency is an emergent 
relationship from the unfolding of transactions between humans and 
object materiality (Tholander et a/., 2012), for instance in the case of 
“Ultra-Surfaces” developed by a MADEC team (Ferrara and Pasetti, 
2020). Ultra-Surfaces ability to perform dynamic behaviors while in- 
teracting depends on their phygital materiality, applied in specific con- 
figurations. It appears that they have a form of agency that allows them 
to achieve a certain goal while developing closer relationships with 
users. The user’s actions and the object’s materiality together contrib- 
ute to a unique relational experience. 

We consider this particularly important for what we will examine 
in depth later: the behavioral and animacy aspects of animate objects. 


6.2 The discourse on objects’ smartness 


Since computing and the digital revolution entered everyday life in 
the late 80s, concepts such as “ubiquitous computing" (Weiser, 1993), 
“Internet of things” (Ashton, 2009), and SPIMES (Sterling, 2005) 
have stimulated reflection on scenarios about features of spaces and 
artifacts. Recently, Mike Kuvniasky (2010) has talked about “smart 
things”, things that are embedded with smartness or intelligence, iden- 
tification, automation, monitoring, and controlling functions. 

In the industrial design community, the issue of objects’ evolution 
has been faced by the scholars Ezio Manzini (1990) and Giovanni An- 
ceschi (1993), who added to the literature on “Ambient Intelligence” 
(Aarts and Marzano, 2003). 

While Manzini developed a theoretical reflection on “cognitive ar- 
tifacts” (1990), Anceschi (1993) developed a reflection on interfaces. 
For Manzini and Anceschi, the fundamental transformation affecting 
design is the entry of the fourth dimension: time. According to Man- 
zini (1990), to design the interactivity means 


dare al tempo il valore di parametro organizzatore [...] occuparsi dell interattività 
significa dunque avventurarsi su un territorio del tutto nuovo, 1 cui riferimenti 


99 


Copyright © 2021 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788835125716 


culturali vengono [...] dal cinema, dal teatro, dalla musica: da attività, cioè, in cui la 
qualità si produce organizzando sequenze di eventi (Manzini, 1990, p.140).? 


Together with time, Manzini identifies another relevant characteri- 

stic: «Il modo in cui gli attori coinvolti nella generazione dell’evento 
collaborano per raggiungere il risultato previsto». This vision seems 
to us to successfully frame the concept of agency as an emergent rela- 
tionship from transactions between humans and objects. 
The Ambient Intelligence vision emphasized the qualitative features 
of sensitive, responsive, intelligent scenarios in order to support the 
design of user-friendly and unobtrusive artifacts for humans in their 
everyday lives, offering at the same time a more fulfilling user expe- 
rience. Similarly, the literature on “Calm Technology” (Weiser and 
Brown, 1997) has contribute to spread an approach to humanly fit 
technology. 

In the late 80s, the HCI and Interaction designers community began 
to address interactivity by focusing firstly on technical aspects and 
then on usability. Recently, several authors have shifted their focus 
toward an “aesthetics turn”. The contribution of Donald Norman 
(2002)? is one of the most incisive in overcoming the paradigm of ef- 
ficiency and functionality by including reflections on usability and 
also emotions, feelings, relationships, and psychological and semiotic 
aspects of objects and their behaviors, all as elements that improve 
usability. 

The current “material turn” in interaction (Robles and Wiberg, 
2010), related to material interfaces, has accelerated a return to basic 
design (Hallnas, 2011) as well as a move toward “pragmatic aesthet- 
ics”. The latter allows researchers to focus on the aesthetics of behav- 
iors, the finality of design, and new artifacts (Russo and Ferrara, 


? Translation: «giving time the value of organizing parameter [...] dealing with interactivity 
means therefore venturing into an entirely new territory, whose cultural references come [...] 
from cinema, theater, music: from activities, that is, where quality is produced by organizing 
sequences of events». 

3 Translation: «the way in which the actors involved in the generation of the event collaborate 
to reach the expected result». 

4 Norman states «attractive things make people feel good, which in turn makes them think 
more creatively. How does that make something easier to use? Simple, by making it easier for 
people to find solutions to the problems they encounter. 
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2017). In this scenario, we think it is time to strengthen design-led 
research on phygital design, because the quality of any object cannot 
disregard those design aspects inherent to their materiality, nor the cul- 
tural, social, and ethical dimensions that are at the basis of objects’ 
meanings and values. 


6.3 Animate objects: an attempt at definition 


Coming from the Italian Design Culture that has developed in a 
strict interplay between humanities and technology, we choose to 
adopt the term animate objects for those emerging intelligent everyday 
objects that are able to perform behaviors thanks to their phygital ma- 
teriality. 

The adjective animate was first used by Anceschi in his previously 
cited book to describe the fundamental novelty of “colloquial objects” 
(p. 9), i.e. objects’ «new ability to interact with us»? (Anceschi, 1993). 
Highlighting the phenomenon of new technical objects’ status and 
abilities, Anceschi adopted a bio-constructive perspective, the same 
applied to the development of artificial intelligence with reference to 
the intelligence of nature. In line with this perspective, we also use the 
term “animate”, which derives from the Latin animat and indicates 
something «alive or having a life». Already, in a previous study, we 
used the expression “almost-living objects” to define the same cate- 
gory of everyday objects implemented with emerging technologies. 
These are «capable to somehow interact with human beings, determin- 
ing interesting connections and emotional responses in terms of what 
can be defined as a sub-branch of the applied aesthetics domain, smart 
aesthetics» (Russo and Ferrara, 2017). 

With their “behavior”, animate objects modify the relational space, 
modulating their connection by following users or other objects. 

Through analyses of case studies, including those that we will pre- 
sent later on, we have identified and characterized animate objects 


> Anceschi derived this term from anthropologist Franco La Cecla’s (1991) neo-animist the- 
ory of objects. 
6 From the Oxford dictionary. 
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according to their main features. This allows us to highlight which as- 
pects we believe must be taken into consideration when approaching 
their design. 

e Shapeshifting 

This is the capability to change state and appearance thanks to 

phygital materiality. The activation of light signals, sounds, colors, 

temperature, movements, and even smells are changes that show 
the capability to respond to stimuli and interact. 
e Data processing 

This is the capability related to the Internet and computational pro- 

cesses. Animate objects can, through the Internet, collect, process, 

and exchange data and information, affecting objects’, and conse- 
quently users’, behaviors. 
e Independence and pro-activity 

A certain degree of independence and pro-activity contributes to 

animation. This is critical to shifting from a purely functionalist and 

passive view of objects’ performing a specific function toward an 
active behavioral function view. 

These capabilities contribute to objects’ shaping, expressing, and 
performing behaviors, and stimulating responses to users or other ob- 
jects which in turn can further influence users and stimulate new be- 
haviors. These are the fundamentals for a smart experience and aes- 
thetics. 


6.4 The role of aesthetics in the design of animate objects 


According to the designer Yves Béhar: 


aesthetics is the support of experience both from the appearance and the functional 
side of an object, driving one’s eyes or hands toward the product so as to strengthen 
the temporal and structural logic of the project (Béhar, 2020, p. 234). 


In the last decade, the scholar Lars Hallnas (2011) has launched a 
proposal for a revision of the aesthetics of interaction at a fundamental 
level. He suggests taking a step back from the perspectives of experi- 
ential design and design for emotion, and to revisit the elementary no- 
tions of form and expression based on design fundamentals (form, 
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color, texture, sound, smell, and behavior), rather than focusing on be- 
havioral and social sciences as is common in HCI. He relies on an 
approach that focuses on the “act of use”, “interface”, and «processes 
of learning how to use given things», with strong relation «function 
and interaction in space and in time». He defines the central design 
dimensions for the classification of interaction design variables as the 
following: 

e Timing - the rhythm and meter of use we introduce. 

* Spacing - the space of use we introduce. 

* Connectivity - the connections of use we introduce. 

e Methodology - the ways of use we introduce (Hallnas, 2011). 

Other scholars maintain a double focus on expression and experi- 
ence, meaning the aesthetics of interaction remains somewhat differ- 
ent from the aesthetics of traditional product design (Petersen et al., 
2008). Thus, despite the abundance of studies, HCI has not yet been 
able to achieve a clear and convincing aesthetic perspective. 

We believe that today it is essential to recover a basic design ap- 
proach. We also believe that it is important to articulate a perspective 
capable of incorporating contributions from other disciplines to struc- 
ture design purposes in terms of method and ethics. 

Currently, one of the most respected perspectives on aesthetics is 
Richard Shusterman’s pragmatism. Briefly, an “aesthetic experience” 
has a practical use as well as an intrinsic value. It «invigorates and 
vitalizes us and thus helps us achieve the ends we pursue» (Shuster- 
man, 2000). It relates to a user’s experiences and is linked to the socio- 
cultural context because the meanings and values we attribute to ob- 
jects constantly change as cultural groups change. It has an ethical and 
social dimension. 

Form, which is “a dynamic interaction of elements” (Shusterman, 
2008), is closely related to the aesthetic experience and is strongly 
linked to the body (its senses and movement) and the social conditions 
that helped structure it. Finally, Shusterman’s “somaesthetics”” puts 
the body and its movements back at the center as fundamental parts of 
feeling and thinking (Shusterman, 2011). The human participates and 
structures the aesthetic experience with his body and mind. 


7 Somaesthetics combines the terms “soma” and “aesthetics”. 
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These concepts are particularly relevant for the design of animate 
objects because they reveal the wide potential for interacting with 
them. Their shapeshifting in time makes them agents of “kinesthetic 
interaction” (Fogtmann et al., 2008), including multisensory stimula- 
tion and excitement, movability, implicit and explicit motivation, and 
empathic response related to the body. Emotional, affective, social, 
and cultural stimuli are added to the body stimulation process, leading 
to cognition, reasoning, and judgment. 


6.5 The behavior of animate objects 


In neuropsychology, behaviors are the externalizations of emo- 
tional processes, associated with bodily movement and certain orches- 
trations of reactions to given causes, within a given environment 
(Damasio, 2000). Regarding objects, behavior is related to action and 
reaction, to what happens in a particular context, and to a change of 
state or a movement in time that occurs in the object itself. We refer 
to behavior as an orchestration of dynamic sequences of physical 
movements or a change of state in the material world (in form, color, 
light, density, sound, etc.), making an explicit link between form and 
expression, the abilities to stimulate relationships with users and create 
specific experiences. 

In biology, a behavior is a neural response that results in a stimu- 
lation of the senses and an effect on motor activity. 

We can then define the behavior of an animate object as the pro- 
duction of transformations that we can observe and promote or that are 
implemented in response to an environmental change - but always 
self-generated toward a specific goal or as a reaction to an external 
event. 

Animate objects are sources of change through their behavior ac- 
cording to a principle of intentionality, i.e., a design. 

On the behavior of objects, there is also the recent literature on “be- 
havioral objects” that connects art, robotics, and cognitive psychology 
to emphasize a double point of view: the agency of behavioral objects 
and the cognitive faculties of the observer (Bianchini et al., 2016; 
Levillain and Zimbetti, 2017). Firstly, this focus recalls 
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anthropomorphism, i. e. the innate tendency to readily attribute formal 
and mental states (emotional, intentional, behavioral, etc.) to living 
and nonliving entities. Animate objects, having the ability to interact 
with the external world, suggest living-being-like (anthropomorphic, 
zoomorphic, or even plant-like) traits to an observer, as well as mental 
traits. When this happens, the relation of the user with the non-living 
entity manifests an affective state. 

We believe that the behavioral characteristics of animate objects 
must be studied and designed in the light of physical variables that 
relate to changes of state (form, dimension, density, light and color 
effects, movement, etc.). There are already some guiding principles 
around this, for instance regarding movement. 

The “12 Principles of Animation”, which were developed by Walt 
Disney Studio in the 1930s (Johnston and Thomas, 1995), are still a 
valid tool for designing movement as transition from one shape to an- 
other, adhering to basic laws of physics as well as emotional timing 
and character appeal. More recently, Strohmeier et al. (2016) have ex- 
plored the link between the shapes of a flexible surface and emotions, 
demonstrating that movement might be used to convey messages with 
emotional content. Similarly, design guidelines can be formulated for 
light and color shape changing. 

Secondly, the behavioral focus recalls the human capability for sig- 
nification and assignment, known as “attribution” and defined in so- 
cial psychology as the faculty of being able to explain or infer a be- 
havior (Malle, 2011). Through attribution, humans are able to accu- 
rately assign and predict certain psychological attributes to external 
entities which then determine specific social characteristics such as 
intention, motivation, and purpose. 

At a deeper level then touching the affective aspects related to be- 
havior, it is necessary to mention our tendency as observers to place 
ourselves in an empathic relationship with others, that is, our ability to 
understand and respond to the emotional states of other agents. 
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6.6 Exploring the behavior of animated objects: case studies 


After defining animate objects and describing our thoughts on the 
existing literature, we move to the current scenario of research prac- 
tices to analyze some case studies that fall under this new class of ob- 
jects. The chosen case studies contain the features we previously iden- 
tified. The projects were analyzed including interviews with the de- 
signers. We have reported on aspects such as the technologies and ma- 
terials applied, the form and expressivity, the behaviors, as well as the 
designers’ intentions and narratives. Moving beyond pure appearance, 
we have identified these objects as agents of smart aesthetic experi- 
ences, which include augmented perception, relational communica- 
tion, and empathy (Russo and Ferrara, 2017). 


6.6.1 Lift by the design studio Patten Studio, 2017 


Let’s take a close look at Lift? (Fig. 6.1) designed and produced by 
James Patten and his team. This is an interactive, heat-sensitive, mo- 
torless light feature consisting of 24 LED petals attached to a spine. 
Each petal embeds a microcontroller that receives data from one of six 
thermal imaging cameras (infrared sensors) in Lift’s ceiling mount. 

Lift is a good example of system design that applies a smart mate- 
rial (wires of Nitinol, a shape memory alloy commonly called muscle 
thread) and microelectronics jointly to consolidate materials (metals 
and polymers for the lamp body). 

Moreover, it is a good example of smart aesthetics. This animate 
object has a high expressiveness, mediating animacy through its com- 
putational power and material performance. Lift’s petals respond to 
human presence with a movement that looks more natural than me- 
chanical in terms of fluidity and timing. Each petal works in a coordi- 
nated way with the others: it reacts to nearby people by fluttering 
slightly if someone walks beneath it or moving more dramatically in 
response to wild movements. In these instances, the closer petals move 


8 Lift received the Core77 Award 2017 in the Furniture & Lighting category. 
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Fig. 6.1 - Lift, interactive light by Patten Studio. Courtesy Patten Studio. 


away, as if communicating a feeling of fear. The long central spine of 
the lamp is composed of pieces like vertebrae, and extends smoothly 
and asymmetrically, like a snake, resulting in a fluid, organic-looking, 
and completely silent movement imbuing spaces. 


With some of the richness, nuance, and immediacy - the same we would expect 
from the natural world. [...] It will respond as long as those people continue to move 
around. If those people stop moving, it will gradually return to an idle state, even if 
they do not leave the room. 

Essentially, Lift is seeing human activity by noticing changes in heat in space. 
[...] Environments that acknowledge our presence within them help us to relax - to 
feel that we belong. Interactivity engages us, helping us to tune into our surround- 
ings, embrace the present moment, and connect to the people we share physical space 
with (Patten Studio, 2021)”. 


9 From the interview with James Patten. 
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In Lift’s behavior, we recognize the will to engage the observer in 
a kinesthetic experience. The smooth shape of the spine from one side 
to another involves the observer under the petals, who follows the flu- 
ent curves and movement. Like a vegetal organism turning its head 
toward the sun, Lift orients itself toward the people entering its envi- 
ronment (Pattern Studio, 2017). 


6.6.2 Opale by Behnaz Farahi, 2017 


Opale is a shape-changing interface, also defined as an “emotive 
garment” (Farahi, 2018) (Fig. 6.2). The focus of this project is the dy- 
namics of social interactions. Opale’s behavior is based on its recog- 
nition of the basic human emotions that are considered universal and 
physiologically associated with facial expressions. We normally re- 
spond intensely to the expression of an onlooker by adapting our emo- 
tional expressions to others’ (whether happiness, sadness, surprise, an- 
ger, or a neutral expression). This is a neuropsychological process led 
by mirroring neurons that works as a communication channel. 


The design challenge was to critically explore whether emotions expressed in 
our social interactions could be represented in a non-verbal way through the motion 
of a garment (Farahi, 2018). 


In technical terms, Opal uses a facial tracking camera that captures 
the facial expressions of people around and a microcontroller to acti- 
vate a mechanical system. It is equipped with a pneumatic actuation 
system capable of generating patterns and various inflation speeds to 
mimic emotions, thanks to small inflatable silicone pockets. On top of 
the pockets, thousands of thin fibers are embedded and move by fol- 
lowing the pockets’ expansions. The movement is inspired by the be- 
havior of animal fur, to which the designer dedicated an attentive study 
so as to accurately decide the location and orientation of fibers and 
shape an effect not dissimilar to nature. Subsequently, the fiber distri- 
bution on the wearable surface was adapted to the biological structure 
of the human body and refined in density and weight to make the effect 
more realistic. The object’s behavior-changing creates an aesthetic- 
emotional experience and influences the behavior of the observer. 
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Creating a system of exchange «the material then responds physically 
to the detected emotion in order to establish an effective loop with 
users» (Farahi, 2017 and 2018). 


Fig. 6.2 - Opale by Behnaz Farahi, 2017. Courtesy Behnaz Farahi. 


6.6.3 Pinokio by Shanshan Zhou and Adam Ben-Dror, 2012 


Pinokio (Fig. 6.3) is an experimental attempt to transform a com- 
mon artifact into an animated object by designing a behavior through 
movement. Designed by Shanshan Zhou and Adam Ben-Dror, an an- 
glepoise-type table lamp uses a wide camera to track a user’s move- 
ments, which are then transformed into the object’s movements via a 
microcontroller. Conceptually, the two designers used body storming 
and role-playing to analyze and understand the object’s possible 
movements. They then translated the desired behaviors into move- 
ments in the joints of Pinokio’s arm using inverse IK kinematics!®. 


The central concept is to explore if we can evoke empathy in humans when they 
interact with a machine. We wanted to subvert our “relationship” with a tool from 
being purely functional, to be empathetic and emotional instead. We wanted to 


10 This is a technique that uses trigonometry to define the spatial movement of multiple parts, 
commonly used in animation and robotics. 
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experiment and explore which type of interaction can help to develop a connection 
between a human and a machine (Zou and Ben-Dro, 2021)!!. 


Fig. 6.3 - Pinokio, 2012. Courtesy Shanshan Zhou and Adam Ben-Dror. 


Object and user find themselves equal, as the movement of one in- 
fluences that of the other (Ben-Dro, 2012). The designers add: 


The behaviors we worked on were very simple, they are just enough to give you 
a sense that this character has some personality, and we didn’t want to create a nar- 
rative because then it becomes deterministic and more like animatronics that plays a 
routine. 

We worked on simple behaviors such as curiosity (looking around randomly), 
research (looking for faces, and focusing on people’s faces), shyness when people 
look at Pinokio too long (looks away, ducks down as if to hide), tiredness or boredom 
(looks down lazily), relaxing (turns off the switch, covers the switch). The key to 
this aliveness comes down to Pinokio’s interaction with humans and the environ- 
ment: a living creature that responds to random events in the environment (Zou and 
Ben-Dro, 2021). 


11 From the interview with Shanshan Zhou and Adam Ben-Dror. 
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6.6.4 Addicted Products, by Simone Rebaudengo with 
Haque Design + Research, 2012 


The last project we are going to present explores gestures and the 
implication of animate objects being connected. Like in the case of 
Pinokio, Addicted Products (Fig. 6.4) applies technologies to common 
objects like toasters, lamps, etc. Using a microprocessor and an Inter- 
net connection, the objects become animated and capable of providing 
a service. The designer imagines a series of toasters connected to a 
network making decisions autonomously: sending the shopping list to 
the nearest store, communicating with each other by exchanging in- 
formation about their use, and even deciding to look for a new user by 
posting a message on a social network, if they are not suitably used. 
The designer explains: 


People would apply to an online site for the opportunity to host a toaster, and 
then the objects themselves would decide where to live based on their perception of 
being used enough. By being connected their perception of being used, a sort of 
rudimentary happiness from an object’s perspective, was influenced by all the other 
toasters in the network, creating a sort of peer pressure for objects. Based on their 
happiness I then programmed a series of behaviors, from a simple toggle movement 
to attract attention, to tweets to complain to the most extreme behavior for an object: 
breaking the user/object bond and asking a better user/object to leave and sending a 
messenger to take it to its new and potentially happier home (Rebaudengo, 2021)!?. 


Addicted Products become social agents. Through the movements 
of the toaster, the user can assign an initial series of behavioral param- 
eters to which are then added all the other relational capacities of the 
object that manage to influence, not only the nearby environment, but 
other agents (Rebaudengo, 2012). 


12 From the interview with Simone Rebaudengo. 
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Fig. 6.4 - Addicted Products by Simone Rebaudengo with Haque Design+ Research, 
2012. Courtesy Simone Rebaudengo. 


6.7 Discussion 


From the analysis of the case studies, taking into account the design 
intention, the user’s involvement, and the context in which the objects 
act, a variety of aesthetic experiences emerge. The relational experi- 
ences between user and animate objects mainly refer to: 

e The sensory-motion dynamic that happens when behavior primarily 
affects the senses and the body, stimulating perception, motion, and 
motivation. 

* The sensory-emotion dynamic that overcomes the body stimulation 
with an empathic relation and emotions coming from the process of 
attribution, and mainly depends on the coordination of shape-changing 
in time and space. 

e The cognitive-behavioral dynamic, which no longer concerns just the 
relationship with the object through motion, but also affects cognitive 
processes. 

e The ethical dynamic related to sense-making and reasoning that ar- 
rive at ethical judgments. 


The dynamics presented are actually various traits of the entire user 


experience, which contains sensory and ethical traits on opposite sides. 
In fact, the aesthetic experience is like a continuum in which those 
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dynamics are all present to different degrees, creating an impact at an 
emotional and a cognitive level, encoding new attitudes, feedback, 
gestures, and communicative issues. 

A similar distinction can be made with regard to the spheres in- 
volved, namely the individual and the social. 

In the individual sphere the process of adapting behaviors and the 
relational experience are primarily binary between users and objects 
and vice-versa. The social sphere involves multiple agents, both hu- 
man and other animate objects. The process of adapting behaviors is 
no longer binary but relates to a wide network. 

For instance, in the case of Lift and Pinokio, the aesthetic experi- 
ence involves sensory-motor and sensory-emotion dynamics in the 
surrounding space, although in different ways. 

Lift richly occupies the surrounding space with its presence and 
behavior, interacting with the observer, who is quickly emotionally 
caught up in the game between fluid forms and the coordinated time 
of movements. The kinetics of Lift is linked to that of people in the 
surrounding space. Each petal responds by moving according to the 
movement of the observers and reacting sequentially. The design of 
the long central spine sinuously accompanies the movement of the pet- 
als and distributes them within the space. Lift seems to understand and 
communicate emotions of happiness and fear through its petals’ ki- 
netic speed. Shapes and movements together recall natural organisms 
and provoke empathy in the user, who becomes almost a playmate. 

Pinokio involves the user’s personal space in a more invasive way, 
acting out a binary sensory-motor dynamics. The object, initially 
quiet, starts to move when turned on. Its kinetisms appear like a wak- 
ing-up and then a seeking of an intimate relationship with the user, 
who is also influenced by the object’s animacy. Moreover, Pinokio 
expresses curiosity about its user and even demands attention, modu- 
lating movements through its articulated arms and the rotation of its 
head. It extends and contracts, depending on the proximity of the 
user’s face. Even in this case, shape-changings create an empathic re- 
sponse from a user who looks like a shy and curious playmate. 

In the case of Opale and Addicted Products, a critical intent in- 
volves the body, mind, and society. In these projects the cognitive 
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dynamics seem to be stronger, and the involvement shifts from the in- 
dividual sphere of a user toward a social sphere. 

Emotions seen as social dynamics are the focus of Opale. The de- 
signer reflects on the effects of our unconscious facial expression and 
our body movements related to emotions and imagines a wearable ap- 
pendix that can empathize with the onlookers. 

Emotions are detected and expressed by the object, and through 
them, Opale involves the user and people around in a behavioral loop 
with their bodily reactions and, consequently, their minds. Movement 
depends on perceived emotions and their expression in turn influences 
movement again. 

In the Addicted Products experiment, we recognize a strong ethical- 
social dynamic. In this project, the designer imagines a society in 
which there are self-motivated objects with independence, considering 
both the positive and problematic aspects. The objects’ behavior con- 
sists in gesture-like movement able to communicate emotion to users, 
such as moving its lever frantically to complain. This makes objects 
capable of organizing themselves to respond to problems of consump- 
tion and circularity. The designer uses them to try to give an answer to 
a social and ethical problem, that of production and waste. We add a 
scheme to visualize the relational dynamics that characterize the aes- 
thetic experience with animate objects referred to in the four case stud- 
ies (Fig. 6.5). 


6.8 Conclusions 


Through the analysis of the aforementioned theories and case stud- 
ies, it has been possible to define the new class of animate objects and 
characterize behavior as a new parameter for reframing objects’ smart- 
ness in the ongoing process of phygitalization of our ambient. 

Animate objects, with their capability for shapeshifting, processing 
data, independence, and pro-activity, express behaviors. These di- 
rectly influence human behaviors, including our perception, attention, 
motivation, and, in general, our decision-making capabilities. 
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Fig. 6.5 - Scheme of relational dynamics that characterize the aesthetic experience 
with our four chosen animate objects. 


This means that they both stimulate our senses and our cognition, 
establishing relational qualities in which objects’ meaning and value 
are built. 

Through the case studies, we tried to offer a phenomenological 
taste, albeit minimal, of the current research practices on animate ob- 
jects. These include common objects (toasters, lamps, clothes, etc.) 
with a primary function accompanied by a second one that adds qual- 
ities to the experience resulting from the object’s presence and use. 

This scenario is comparable to neither computers nor robots in 
terms of complexity of the technical functionality or efficiency of the 
tasks. From Alain Turing’s (Turing, 1950) foresight that machines 
would become more effective if they became credible companions, we 
shift to the philosophy of “meaningful presence”, in which common 
everyday objects become machines to improve our lives from an eco- 
logical point of view. Their presence is «part of who we are, how we 
live and how we express ourselves» (Hallnas and Redstrom, 2002, p. 
121) - and not just. Animate objects carry affective relationships in the 
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everyday experiences through which we assign meaning as well as 
value to life. 

We believe that the aspects presented so far are of considerable im- 
portance, in view of wider reflections on product innovation. They can 
help us understand the phygital evolution of our environment, with its 
strengths but also difficulties, and open possibilities of choice. 

Today’s, objects’ behaviors, jointly with the use of time and space, 
are the central parameters for designing animate objects. The latter 
represent a design challenge that requires further interdisciplinary re- 
search. We believe that the behavioral characteristics of animate ob- 
jects must be explored and designed in the light of physical variables 
that relate to shape-changing, in terms of changes in form, dimension, 
density, light, and color in time. 

We strongly encourage further exploration of behaviors in future 
research. 
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7. Reframing the domestic smartness. Artificial in- 
telligence between utopia and dystopia 
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Abstract 


With AI spreading in domestic environments, the space of the 
house can acquire new meanings, functions and unlock new possibili- 
ties. The digital realm can pervade physical spaces to better take care 
of the inhabitants of the house. However, how are private domains go- 
ing to evolve to embrace the potentialities and changes that AI is of- 
fering and play a crucial role in developing our well-being? 

These practices are at an embryonic phase, and not enough reflec- 
tions are being made in the design community as they are missing the 
tools and methods to deal with AI. Nevertheless, the design discipline 
might translate its expertise into this trending technological field, fos- 
tering concrete answers and reasonable solutions for a new way of in- 
habiting the domestic environment. 

Accordingly, the primary purpose of this chapter is to stimulate a 
conversation about this topic in the design community. To drive the 
investigation and trigger critical reflections, six near future scenarios, 
in the form Science Fiction Prototypes (SFPs), are envisioned. They 
are rooted in the convergence of psychology, design, and AI, revolving 
around theories of emotions to impact human beings beneficially. 
Then, a survey, including the evaluation of four of the SFPs, is pre- 
sented to advance the discussion further and pave the way towards a 
designerly perspective on AI systems for the domestic environment. 
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7.1 Domestic spaces between physical and digital: a possi- 
bility for design 


The time for physical spaces to hybridize with the digital realm has 
already arrived. Smart speakers integrating virtual assistants represent 
the first widespread materialization of AI in the domestic environ- 
ment, and they are a continuously increasing trend. Installed smart 
speakers are globally forecast to reach 640 million by 2024 - doubling 
the 320 million in 2020. Additionally, it is more and more frequent to 
encounter other Al-enhanced objects in our houses: lights, thermo- 
stats, doorbells, and even showers, while robots - varying from domes- 
tic assistants to pet substitutes - are multiplying (Hitti, 2020). Then, 
the road to a paradigm shift in ambient computing is paved. 

However, the public responds with a mix of contrasting sentiments. 
On the one side, there is hype and excitement for novelty. On the other, 
excluding the anxiety generated by filmic worlds overwhelmed by ro- 
bots, disappointment and frustration are the most frequent sentiments 
characterizing the use of smart speakers. The design of these devices 
is one of the main causes: a lack of understanding produced by under- 
lying invisible operations (Norman, 2004), issues in essential elements 
of interaction like discoverability and proper exploitation of functions 
(Kinsella, 2018; White, 2018), and the unfulfilled promises made by 
advertising campaigns make smart speakers miserably fall below ex- 
pectations. 

Indeed, a positive impact of technology on people’s mood and life 
is key: Norman (2004) states that it should bring richness and enjoy- 
ment. Aware of that, brands delivering AI-enhanced products are lev- 
eraging emotional aspects in their commercials. Accordingly, Amazon 
Echo is a tool to connect people or just the evolution of a human mate. 
At the same time, Google, emblematically states “You make a house 
a home. We make a home a nest”, overshadowing the product by high- 
lighting people’s different emotional states at home. 

The reality, though, tells us a very different story. Even if no deep 
investigations addressed smart speakers’ emotional impact, especially 
in the long run, we could intuitively compare them with design theo- 
ries related to (positive) emotions to understand that they need further 
development. As anticipated, following Khaslavsky and Shedroff’s 
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principles of the seductive power of design (1999), they excel in the 
enticement step with attractive promises, but miserably fail in main- 
taining them and in making a memorable experience. Likely, whether 
they respond to an idea of novelty, favorably preserving the quite fa- 
miliar aesthetic of a speaker, they often require some effort to perform 
a task, contradicting the first and overarching beneficial feature Hek- 
kert (2006) recognizes. As well, problems arise when it comes to the 
meaning of the experience, as it is difficult for them to respond to per- 
sonal significance, virtuous (Desmet and Pohlmeyer, 2013), reflective 
(Norman, 2004), psychological, or ideological qualities (Jordan, 
2000), even marking a rejection of sensuous curiosity and pleasure 
(Spallazzo, Sciannamé and Ceconello, 2019). 

These limitations, accentuated by the daily relevance they have in 
the domestic sphere, underline how the materialization of AI repre- 
sents a field full of possibilities for design. For example, one possible 
area for intervention is humanizing a technology that still seems too 
far from people. For that, emotional intelligence could play a relevant 
role. Indeed, as Minsky (2007) states, we need to equip machines with 
different reasoning processes, and emotions can be considered ways 
to think we may use to increase our resourcefulness. 


7.2 Setting the space for a designerly discussion: our 
method 


The conceptual exploration moves from the intersection of design 
and AI with human sciences, psychology in particular, trying to over- 
come the mental restrictions that impede disruptive advancements. 
Different theories about emotions have been adopted as literal guide- 
lines to orientate the design process and stimulate critical thinking as 
a sort of exercise in style. Sharing the spirit that drove Raymond Que- 
neau (1947/2007) to experiment with literary variations of speech, we 
activated the creative engine towards unconventional homes for hu- 
man well-being. Meanwhile, we collected different pictures that por- 
tray alternative futures and embody the core principles of the currents 
of thought they stand for. 
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As a result, the outcomes have been experimentally employed to 
instill a discussion in the design community. 
Below, the two main phases of the investigation are depicted. 


7.2.1 Science Fiction Prototypes 


Future-focused thinking is one of the primary activities of design, 
especially when related to emerging technology research and the HCI 
field. For this purpose, the method of science fiction prototyping has 
been adopted. SFP is considered an embodiment of the third wave of 
design and HCI practices and aims at capturing the essence of culture, 
emotion, and experiences, emphasizing the reflective role of the de- 
signer in HCI processes (Kymäläinen, 2016). According to Dourish 
and Bell (2014), Science Fiction Prototypes (SFPs) as tools for design 
research could not only anticipate but actively shape the technological 
futures. In this context, SFPs are employed according to Keinonen’s 
model of design-oriented research (2006) as artifacts that may “open 
new spaces for design”, triggering critical thoughts and further specu- 
lations about the topic. In a reinterpretation of his model, here the phe- 
nomenon to be analyzed (field of research) is AI in the domestic envi- 
ronment, while the framing for interpretation (lying in the field of psy- 
chology) are positive emotions, conveying focus and creativity to HCI. 
Design, instead, maintains its overlapping role to enable the commu- 
nication between the other fields and brings, as an added value, its 
expertise to deal with the moment of drunkenness characterizing the 
introduction of new technology (Antonelli, 2018). 

The scenario building is based on Johnson's steps to create an SFP 
(2011), which is accordingly articulated in (1) technology enabling the 
scenario and set up of the world, (11) scientific inflection point - which 
dramatic change marks the transition to a new progressive scenario?, 
(iii) implications and ramification of the technology for the world, (iv) 
human inflection point - how people’s life is modified?, (v) implica- 
tions, solution or lessons learned - judgments are intendedly avoided 
not to influence the reader. In the following, SFPs are presented in a 
more narrative style, still covering all the elements. Hence, six scenar- 
ios portray more or less near future Al-enhanced houses: (1) the 
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physiological house, (2) the neurological house; (3) the cognitive 
house, (4) the activity house, (5) the hedonic house, (6) the eudemonic 
house. According to a specific theoretical conception, they are delib- 
erately designed for people’s well-being, explained at the beginning 
of each SFP. In particular: (1), (2), and (3) address the issue in the 
most general sense and from a causal perspective; (5) and (6) express 
possible ways to pursue positive emotions, while (4) lies in-between. 
The outcomes are intentionally hyperbolic to stress their characteris- 
tics, and they are flawed with an individual-centered perspective, re- 
flecting the ego-centric character of the psychological theories. More- 
over, they try to highlight the critical points of the technology, auton- 
omy, adaptivity, and the resulting possibilities. 

1. The physiological house. Commonly accepted theories assert 
that emotions and the physiological responses manifested by human 
bodies are patently interrelated. In some of the earliest emotion theo- 
ries of modern psychology, physiological responses produce emo- 
tional behaviors, as the Peripheral Theory exemplifies (James, 1884); 
or they just co-occur, it is the case of Cannon-Bard Theory (Lindzey, 
Thompson and Spring, 1991). Either way, modifications of skin con- 
ductivity, body temperature, heart rate, rhythm, blood pressure, and 
other possible reactions, if well balanced with contextual information, 
can make emotions measurable. 

From these premises stems the physiological house. Provided with 
a ubiquitous AI system connected to all the smart appliances of the 
house, it can adapt the environment to people’s specific needs. Here, 
house-bands represent the core of the mutual human-home interaction. 
They measure and keep track of people’s physiological parameters, 
send the data to the house’s AI system, and become means of commu- 
nication by displaying messages. A period of training in which the 
house learns to understand how its users show their feelings and how 
they react to different situations by triangulating physiological re- 
sponses with personal and contextual information is inevitable. Then, 
as a caring mother, the house ensures that, once at home, one finds its 
living space perfectly settled: with the right temperature, lights, music, 
just to name a few. Of course, in the beginning, some misunderstand- 
ings might happen. For instance, if one gets scared watching a horror 
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The Physiological House The Neurological House 


The Hedonic House The Eudemonic House 


Fig. 7.1 - SFPs in the form of six future houses. Illustrations by Sara Scianname. 
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movie, the house may turn on the lights and suggest him/her to call 
someone not to stay alone. However, despite the training time and the 
massive amount of power and digital/physical space to store data 
worldwide, physiological houses can be perfectly efficient in tailoring 
themselves for human well-being. 

2. The neurological house. Emotional responses can also be at- 
tributed to the internal activity of the brain. Neurological theories sug- 
gest that emotions arise in the limbic system (Picard, 1997). Activating 
the amygdala, the anterior cingulate, or the prefrontal and somatosen- 
sory cortices (Damasio, 1994), emotions are an observable phenome- 
non: they leave a neural signature, identifiable through fMRI (Func- 
tional Magnetic Resonance Imaging) as a distributed pattern of activ- 
ity (Kassam et al., 2013). 

Then, monitoring and triggering electrical impulses in the brain, 
the neurological house can affect people’s mental wellness. For this 
purpose, the house’s AI system and its host are in a sort of symbiosis, 
connected through minuscule electrode threads implanted in the brain. 
These collect data that the house reads, analyses, and responds to, di- 
rectly stimulating the appropriate part of the brain to reproduce posi- 
tive emotional patterns - similarly to the neurologist Duchenne de 
Boulogne eliciting emotion-related facial expressions by applying fa- 
radic shock on facial muscles (Duchenne, 1876). As a positive state of 
mind is constantly artificially induced through the brain implant, not 
much is needed inside the house to reach well-being. Indeed, furnish- 
ings and decorations become superfluous, and only the survival-re- 
lated equipment finds a place in the neurological house. Not depend- 
ing on material solutions, people’s status is redefined: their finances 
are increased, as superficial goods or services have no longer use; bad 
habits fulfilling secondary needs, like midnight snacks, are a forgotten 
problem, as the right stimulus can quiet the craving; and some health 
problems, like obesity or drug addiction, can be easily prevented. 
Nonetheless, the electrical response to problems and desires hinders 
people from doing anything non-related to subsistence. 

3. The cognitive house. Cognitive theories of emotions indicate 
reasoning as determining emotional behavior in two main ways. On 
the one hand, an event is appraised as beneficial, non-relevant, or 
harmful before a suitable emotion is elicited (Lazarus, 1991); on the 
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other, when we experience an event that causes physiological arousal, 
the resulting emotion depends on the motivation that our cognitive 
process recognizes (Schachter and Singer, 1962). If no explanation can 
be found, we search the environment for clues for labeling the physi- 
ological response. As emerged in an experiment (Schachter and 
Singer, 1962), people who could not understand that physiological re- 
sponses of their bodies depended on epinephrine injections could be 
emotionally manipulated with induced cognition towards euphoria or 
anger. 

Following this logic, the cognitive house’s AI system, equipped 
with cameras and managing all environmental facilities, manipulates 
people’s internal appraisal of events by staging favorable contextual 
conditions if it detects good behaviors or adverse circumstances if it 
spots bad ones. The objective is to parallel actions with positive or 
negative emotions according to whether they enable well-being or not. 
The parameters for the distinction between good and bad are whether 
based on acknowledged notions about people’s health, explicitly cus- 
tomized according to the hosts’ goals and apprehended over time by 
observing their external manifestation of emotions (body language, 
voice tone). 

Then, the house acts like a stubborn and despotic educator. If one 
wants to get in shape, the house can low the temperature and make the 
lights colder to infer discomfort as (s)he eats sweets or turn on ener- 
getic music to celebrate an exercise session. Either if the user per- 
ceives the intentions of the house or not, (s)he is encouraged to follow 
the right path. Though the motivational process does not have imme- 
diate results, people can quit early or endure and reach their goals. In 
any case, AI finding a balance between physical (health) and psycho- 
logical well-being (happiness) is a real challenge. 

4. The activity house. Even if not explicitly, Activity Theory sug- 
gests that emotions proceed from experience, and they are triggered 
by subject-object activities, intended as purposeful and transformative 
(Kaptelinin, 2014). In particular, Csikszentmihalyi (1990) identified 
the highest level of well-being as flow: the condition of being im- 
mersed in an activity balancing one’s skills and perceived challenge. 

The activity house, as the name implies, advocates an active and 
meaningful home life. As a conversational agent, it proactively 
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proposes to people what to do according to their characters, lifestyles, 
objectives, skills, and external events so that they can effortlessly 
achieve happiness. For this purpose, the AI system connected to the 
house should know the owners better than themselves. Hence, it needs 
to be deployed on their devices to unobtrusively monitor what happens 
to the users throughout the day (via microphone, calendar, email, mes- 
sages). Keeping track of the daily routines and learning from the con- 
stant monitoring of the hosts’ behaviors and reactions, the activity 
house can autonomously organize people’s time and encounter their 
procedural preferences (granting complete transparency to planning- 
maniacs or surprising improvisation-lovers). 

At least at home, people will not need to worry about what to do, 
as their customized managing system can suggest relaxing with a yoga 
session after a tough day or trying a new recipe for a personal sense of 
fulfillment. It can also guide the inhabitants throughout their activities 
by warning them in case of wrongdoing and with visual supports. In 
such a house, time-waste is no longer a cause of frustration, but defer- 
ring organizational matters can become indispensable over time. 

5. The hedonic house. Emblematically embodying the hedonistic 
tradition - for which life is a quest for pleasure for pleasure’s sake, 
while negative affect is avoided (Deci and Ryan, 2008) - this house is 
filled with appealing appliances, autonomously performing their du- 
ties and attesting their functioning as an enjoyable event. For instance, 
a laundry machine directly linked to the wardrobe manages dirty and 
clean laundry, deciding when to wash or refresh clothes. While in op- 
eration, it gives off a fresh scent to please people with positive sensa- 
tions and communicate its operational status. 

Of course, space is appropriately configured to facilitate the auton- 
omy of the appliances. Plus, a control room hosts the necessary equip- 
ment for the autonomous (software and hardware) updating of the self- 
sufficient machines. This kind of well-being is ephemeral in nature, as 
demonstrated by the treadmill metaphor (Brickman and Campbell, 
1971): people in constant pursuit of pleasure are like rats on a tread- 
mill. They keep running, but their condition will never change. There- 
fore, as soon as people get used to their functions and positive elicita- 
tions, the home appliances need to evolve, modifying their functions 
and feedbacks. 
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Indeed, the house will be a personal paradise, entirely at the service 
of its inhabitants: it shelters them from burdensome tasks and conveys 
pleasure. If they need something, people just have to ask. It may be 
the next frontier of consumerism. 

6. The eudemonic house. Contrasting with the previous philoso- 
phy, Eudemonic Theories focus on living life in a full and deeply sat- 
isfying way, pursuing personal development and meaning (Deci and 
Ryan, 2008). This conceptualization does not imply that happiness co- 
incides with positive emotions. Instead, it considers well-being as a 
process of fulfilling or realizing one’s true nature, not as an outcome 
or end state. High-level targets are set, including self-acceptance, pos- 
itive relations with others, autonomy, environmental mastery, purpose 
in life, and personal growth (Ryff, 1989). For this reason, the eude- 
monic house must act as a conscience, leaving people total control 
over their choices. 

The integrated AI system is linked with sensors to perceive human 
beings and with enhanced surfaces to communicate with them. In a 
proactive and shared perceptual (Marti, 2010) approach, when the 
house detects the presence of a user, it manifests its interactive inten- 
tion using any available surface and starts it without being prompted. 
The eudemonic house can portray questions, make suggestions or in- 
still reflections, with the only aim to help its inhabitants’ increase their 
personal values. The interaction is smooth and subtle, and it depends 
on the user whether to embrace the cues or not. Indeed, impositions 
are senseless as without voluntariness, involvement is impeded, yet 
freedom can thwart the AI system’s efforts. 


7.2.3 The survey 


Subsequently to their generation, four of the SFPs have been sub- 
mitted to a sample of 12 design researchers from the Department of 
Design of Politecnico di Milano (selected as experts in different areas 
of interest: technology, well-being, spatial or critical and future re- 
lated), in the form of an online questionnaire. This preliminary re- 
search is chiefly intended to gather ideas from a design-oriented 
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perspective about how AI can benefit the domestic environment and 
evaluate the performance of the SFPs coherently with that purpose. 

Primarily due to a reasonable estimate for completing the question- 
naire in under 15 minutes, two SFPs, namely the neurological and the 
cognitive house, were not included in the selection. Both were ex- 
pected to elicit reservations from respondents as the former does not 
present rich stimuli in terms of spatial design, while the latter is not of 
immediate understanding. The others, instead, portray a variety of de- 
sign possibilities, each insisting on peculiar data collection and inter- 
action modalities. 

The survey is structured on three main streams of information (1) 
defining the current position of designers/researchers towards the 
spreading of AI, (ii) assessing the SFPs, (iii) deriving designerly pro- 
jections. In this context, no practical design repercussion has been en- 
couraged: the aim was to identify designers’ concerns on the topic fol- 
lowing Desmet and Hekkert’s model (2001), according to which con- 
cerns can be delineated as attitude (personal disposition), goals (what 
they want), and standards (how they expect things to be). 

(i) To learn how the respondents are disposed toward AI, a straight- 
forward question opens the questionnaire. It distinguishes four possi- 
ble attitudes: digital utopians, beneficial-AI supporters, techno-skep- 
tics, and prophets of doom, adapting Tegmark’s categorization (2017) 
with the addition of a more pessimist position. Moreover, further qual- 
itative indications can be inferred from all the answers. 

(11) To gather the most immediate reactions, the SFPs’ evaluation 
articulates on two different levels: the perceived cognitive appraisal of 
the scenario and the elicited affective response. First, the respondents 
are asked to distinguish whether the imagined house represents a uto- 
pia, a starting point to do better, the premise for deviousness, or a dys- 
topia, allowing the inference of the designer’s major concerns 
throughout the entire set of appraisals. Then, an affective evaluation is 
required. The research’s thematic connotation drove this choice along 
with the fact that it is largely considered effective in providing an in- 
stinctive and mutually understandable assessment of such a complex 
subject. To this end, the model of the Emocards (Desmet, Overbeeke 
and Tax, 2001) has been selected out of the analysis of several emo- 
tion-related UX evaluation methods (Spallazzo et al., 2021). It is based 
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on the circumplex model (Russell, 1980), representing emotions on a 
bidimensional scale, with valence/pleasantness on the horizontal axis 
and arousal on the vertical one. This categorization, together with the 
figurative representation of the emotional expressions, saves respond- 
ents the trouble of verbalizing their affective response: a commonly 
recognized challenging task requiring cognitive involvement, which 
may influence the response itself (Desmet, Overbeeke and Tax, 2001). 
At the same time, it does not suggest categorical emotions that can be 
easily perceived as out-of-context, but it offers sufficiently generic yet 
grained possibilities. To better frame the information, a motivation for 
the choice is also required. 

(iii) Finally, broader reflections are encouraged to supplement the 
picture of designers’ concerns about the spreading of AI in the domes- 
tic context. For brevity in such a wide discourse, two main issues are 
brought to attention: the most valuable AI capabilities towards bene- 
ficial outcomes and the fundamental principles to consider in design- 
ing Al-infused products for the domestic environment (this question 
was optional). This section is intended to collect indications about the 
respondents’ goals and standards. Consequently, it represents an ef- 
fective means to outline a preliminary path towards a designerly way 
to develop genuinely beneficial AI systems. 


7.3 Opening a designerly conversation: initial reflections 


The survey reveals that the solicited designers share an overall pos- 
itive attitude towards AI (i). The direct question shows that 10 out of 
12 respondents agree that “AI may be beneficial, but crucial issues 
need to be addressed in advance” (beneficial-AI supporters). One sus- 
tains that “a new, enriched and enlightened era is approaching for hu- 
mans” (digital utopian). Instead, the other one openly takes a pessi- 
mistic position: “AI is going to improve but we won’t be able to handle 
unintended negative consequences” (prophet of doom). Starting from 
the latter for closer observation, we can notice linearity in the attitude: 
from a negative perception for the future of AI emerges a very con- 
servative approach. Each scenario is firmly repulsed, and only a slight 
improvement of current technologies is an acceptable future 
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development, denoting a strict closure of mind towards the topic. On 
the diametrically opposite side, the digital utopian’s thought evolves 
throughout the questionnaire. Mainly maintaining a positive attitude, 
(s)he develops critical angles and sets boundaries that should not be 
crossed. Finally, within the most accredited and cautious attitude, di- 
versity seems to be the predominant feature. Each SFP is carefully 
evaluated, with no recurrent assumption influencing the judgment. In 
this way, every scenario allows raising different considerations and, 
eventually, a nuanced picture can be framed. Critical thinking unites 
those who positively perceive AI as an asset for the future, a quality 
generally recognized as distinctive for designers. 

Going into the assessment of the SFPs (ii), interesting insights can 
be drawn by analyzing the three required contributes (cognitive ap- 
praisal, affective response, and motivation) separately and then con- 
fronting them to spot discrepancies. Due to the conciseness of the re- 
sponses and the limited number of respondents, this evaluation is 
purely qualitative yet systematic. 


PHYSIOLOGICAL HOUSE ACTIVITY HOUSE 
Utopia NI Utopia NN 
Starting point Starting point 
to do better |M to do better IE 
Premise for Premise for i | 
deviousness = deviousness 
Dystopia E Dystopia 
2 2. 3. 4 
HEDONIC HOUSE EUDEMONIC HOUSE 
Utopia IN Utopia II 
Starting point Starting point 
to do better ME to do better | 
a, I i === 
deviousness deviousness 
Dystopia Dystopia M 
a E T S AAA 


Fig. 7.2 - Cognitive appraisal of the SFPs. 
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7.3.1 The physiological house: a balanced solution 


The physiological house has the most balanced distribution of pos- 
itive and negative opinions, and within these factions, the positions are 
convincingly emphasized. If the general judgment tends towards a 
constructive vision of the system adapting the environment according 
to people’s physiological parameters, the emotional reactions reach an 
equilibrium. On the positive side, the house is perceived as an im- 
provement. It is praised as great, wonderful, helpful, primarily because 
of its capability to adapt - matching personal needs and desires - and 
to automate tasks, bringing comfort and saving time. Some concerns 
about the preservation of human agency and data protection arise. On 
the other, negative impressions primarily revolve around the concept 
of autonomy, intended both as a harmful capability for the AI system 
and as a lack of control for people. Additional comments mark pri- 
vacy, transparency, and non-necessity as problematic issues. Proba- 
bly, the ambivalent perception depends on the fact that the house deals 
with personal data, but without interfering in the inhabitants’ private 
lives, it just manages contextual conditions. Then, the position of re- 
spondent may interchangeably focus on the convenience or the auton- 
omy of the system. 


E Affective evaluation 


| l | E Explanation 


Fig. 7.3 - Qualitative categorization of the physiological house’s assessments. 


7.3.2 The activity house: practical but despotic 
The activity house, instead, sets the beginning of a general tendency 


towards unfavorable assessments. Already slightly intelligible from 
the cognitive discrimination (even if the responses are quite assorted), 
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a predominant uneasiness is manifested by the affective reactions and 
explanations. Positive statements are more tepid than the previous, 
with less enthusiastic terms depicting the house as pleasant, likable, 
interesting, and helpful. What has been chiefly endorsed is the practi- 
cal aspect it embeds: it grants a better organization and possible bene- 
ficial health implications. Inevitably, though, criticisms concern the 
obtrusiveness of the system and the dependency it may instill. The 
same issues are reinforced in the negative observations. Human dom- 
inance and freedom are defended against succumbing to a trespassing 
technology: it makes people “feel watched” and question the meaning 
of being human, which, according to the respondents, cannot be de- 
fined in an exclusively pragmatic perspective. In this case, the AI sys- 
tems need to intrude people’s lives, with the activity log being the only 
source for it to be an effective counselor. Surprisingly, for design ac- 
ademics exposed to the discussion about privacy and trustworthiness 
issues, what bothered them the most is the feeling of deprivation of a 
controlling role. 


E Cognitive 
E Affective evaluation 


| E Explanation 


dil wil 


Fig. 7.4 - Qualitative categorization of the activity house’s assessments. 


7.3.3 The hedonic house: too much 


The sensation is amplified in the hedonic house. Mainly appearing 
as a dystopia, it finds quite a united hostile front. The reaction is almost 
indescribable, with more than one respondent emblematically stating 
“it is too much”. Again, the figuration of this house is rejected, as it 
could take pieces of humanity away. Major fears are related to the au- 
tonomy of the person over machines and the isolation and dissatisfac- 
tion that a compliant behavior may produce. In return, responses bring 
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out a will of escape from the house itself as it is no place to find enter- 
tainment, and the word “afraid” appears twice. Two are the positive 
comments on the hedonic house, appreciated for avoiding burdensome 
tasks but still arising questions about the actual necessity of a func- 
tional and unconventionally entertaining system. Indeed, here the 
practical aspect of the self-sufficient house seems to pass overlooked, 
and just the protagonist behavior, optimized to provide pleasure, is on 
the spot and not gaining popularity. 


E Affective evaluation 


Fig. 7.5 - Qualitative categorization of the hedonic house’s assessments. 


7.3.4 The eudemonic house: intruding human life 


Ultimately, the eudemonic house shows an apparent balance both 
in the cognitive appraisal and affective evaluation, but this is betrayed 
in the more articulated explanations, leaning, once again, towards a 
negative criticism. In the few positive opinions, the tone is still cool 
and the responses are scarcely eloquent. Only one envisions positive 
repercussions for (physical) health, while human agency and caution 
about not confining oneself to the home are advocated. However, they 
portray the house in terms like good, interesting, beneficial, prefera- 
ble, useful, and non-intrusive. The latter, though, is a controversial 
point as intrusiveness is a recurrent motivation in adverse comments. 
Here, the eudemonic house even causes people to feel invaded in their 
freedom, manipulated, judged, and puppets in the hands of a mediat- 
ing, directing, and managing system. Once again, human independ- 
ence is central, and someone, exhausted, states that (s)he associates 
psycho-physical well-being to nature and open air. Then, no matter 
how much the narrative was underlying the discrete, constructive and 
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supportive role of the house, at this point, a sort of oppression seems 
the predominant impression. However, some usefulness is recognized: 
the system can be pleasant and of great help in specific situations, such 
as managing complex problems and entertaining the masses. 


POSITIVE NEUTRAL NEGATIVE E Cognitive appraisal 


TE E Affective evaluation 


a : 

i | | E Explanation 
5} 

| Î I l 
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Fig. 7.6 - Qualitative categorization of the eudemonic house’s assessments. 
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Fig. 7.7 - Affective evaluation of the SFPs. 
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7.4 Designerly reflections and implications 


Finally, the respondents’ considerations (iii) and concerns are out- 
lined to understand what could drive a designerly reflection on the de- 
velopment of AI systems for the domestic space. 

Lacking a proper explanation of the proposed potentialities of AI 
(automating, enhancing, empowering, specializing, and inspiring), 
they have been interpreted in multiple ways, and the argumentations 
about AI capabilities and principles tend to overlap. Though, when 
envisioning the value of AI systems in everyday life, they are framed 
as supporters of human activities, taking on the burden of demanding, 
unpleasant, long, and repetitive tasks in the light of convenience. The 
core message emerging loudly throughout the questionnaire is the es- 
sentiality for humans to maintain autonomy and control their own 
lives, choices, and the machines themselves. This requirement directly 
prompts an interrogation about the ontological meaning of human ex- 
istence, which, in turn, leads back to the key issue of this experimen- 
tation (How can AI be beneficial for humans? What can humans reach 
maximum well-being? What is the purpose of life?). An escalation 
culminating in a complex philosophical discussion that goes beyond 
the purpose of this work and would demand the participation of dif- 
ferent parties. From a UX standpoint, what needs to be considered in 
the design process and overtly communicated to the final user is hu- 
man agency over products and services integrating AI systems. 

To complete the picture of the beneficial features of AI, one also 
enlists customization, while another amplifies the individual dimen- 
sion in an eco-systemic sense, extending the technological potentiali- 
ties in the service of big challenges humanity has to face: the preser- 
vation of the environment, health, human rights. 

Following the principles that should guide the development of AI 
systems, a responsible approach is mandatory. Ethical behavior, moral 
responsibilities, and attention to privacy are recommended, but they 
appear more like a standard to the responding designers. Issues about 
collecting and using data were only seldom cited while assessing the 
scenarios, suggesting that this is important yet secondary in relation to 
more existential matters. 
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Eventually, also values closer to UX practices have been depicted. 
It is the case of unobtrusiveness (systems should operate without in- 
vading the natural flow of human activities), embodiment (currently, 
AI systems are often invisibly embedded in products and services and 
interaction is poorly designed), adaptivity, and transferability (tech- 
nology should support contemporary tendencies and necessities like 
nomadism and temporary housing, pushing customization to a next 
level). 

An additional, curious inferring from the survey is the personifica- 
tion of the AI systems described in the SFPs. They are frequently re- 
ferred to as agents: “artificial entities”, “not my friends”, “with a form 
of judgment” are some humanized epithets that have been used. Indeed 
it is not distinct from how Russel and Norvig technically address AI 
systems in their well-known textbook (2020), but this has more pro- 
found implications for interaction and UX design. The concept of 
products as living objects with which people have relationships, ex- 
plored by Jordan as the “New Human Factors” paradigm (2000; 2002), 
has much more resonance when dealing with a technology born to 
mimic human reasoning. Personifying inanimate objects is a natural 
inclination, yet a controversial issue when it comes to systems that are 
misleadingly defined as autonomous entities, learning and adapting 
over time. Indeed, this may cause some resistance and urgency to de- 
mote AI systems to a supporting role. 


7.4.1 What the future holds 


Although not quantitatively relevant, this preliminary research rep- 
resents a groundwork for fueling the discussion about the development 
of AI systems from a designerly perspective. The depicted concerns 
can be starting points for further, more practical investigations since 
they can drive the generation of methods and tools for designers to 
deal with this technology. They can also bring fresh insights to the 
decision tables. 

SFPs have also proven effective in triggering reactions and reflec- 
tions, leaving room for new applications in focus groups, workshops, 
or didactic experiences. 
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making, applied to diverse fields of inquiry. Over the years, he coor- 
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The book explores evolving perspectives on furniture, interior, spatial and 
architectural da providing a multifaceted view of how the design disci- 
pline and practice deal with the complex and ever-evolving a bet- 
ween the physical and the digital realms. It explores the new frontiers of 
digitally enhanced design, investigating how computation capabilities im- 
pact the design discipline and designers’ thinking and practice. Today more 
than ever, the design discipline must cope with the need to absorb technical 
skills and dialogue with traditionally distant domains. The core competencies 
in the Human-Computer Interaction field are becoming essential to every 
design branch: the computational power is entering the Là process, mo- 
difying how products and spaces are designed, o they are produced, 
and how they will impact the daily life of users. 

The book explores these novel ones roposing captivating portraits 
of digitally enhanced design possibilities, ait tools and processes to ex- 
pressive potentials. 

Industry 4.0 and traditional craftsmanship hybridize in view of a circular 
and just economy for the furniture sector, imagining new approaches to- 
wards the European Green Deal. The sensing SIRIA are intertwined 
with the materials to create a new form of animated objects, proposing a 
novel design approach beyond the user-centered one. The computational 
power of lighting design tools is entering the complex BIM methodology, 
exploring the problematic integration between the two worlds, and propo- 
sing solutions to support the design activity. Artificial Intelligence reframes 
the domestic landscape, thanks to Science Fiction Scenarios, to stimulate 
reflection on the designer's role in framing utopic/dystopic futures. Finally, 
data drive the design of adaptive morphologies, exploring the context of 
computational design with a conceptual framework and reflecting on how 
robotic design can contribute to architecture. 


FrancoAngeli 


La passione per le conoscenze 


